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Abstract 

The three-node multiterminal lossy source coding problem is investigated. We derive an inner bound 
to the general rate-distortion region of this problem which is a natural extension of the seminal work by 
Kaspi’85 on the interactive two-terminal source coding problem. It is shown that this (rather involved) 
inner bound contains several rate-distortion regions of some relevant source coding settings. In this way, 
besides the non-trivial extension of the interactive two terminal problem, our results can be seen as a 
generalization and hence unification of several previous works in the field. Specializing to particular 
cases we obtain novel rate-distortion regions for several lossy source coding problems. We finish by 
describing some of the open problems and challenges. However, the general three-node multiterminal 
lossy source coding problem seems to offer a formidable mathematical complexity. 

Index Terms 

Multiterminal source coding, Wyner-Ziv, rate-distortion region, Berger-Tung inner bound, interactive 
lossy source coding, distributed lossy source coding. 


The material in this paper was partially published in the IEEE International Symposium on Information Theory, Honolulu, 
Hawaii, USA. June 29 - July 4, 2014 and in the ncIEEE International Symposium on Information Theory, Hong-Kong, China, 
June 14 - June 19, 2015. The work of L. Rey Vega was partially supported by project UBACyT 2002013100751BA. The work 
of P. Piantanida was partially supported by the FP7 Network of Excellence in Wireless COMmunications NEWCOM#. The 
work of A. Hero was partially supported by a DIGITEO Chair from 2008 to 2013 and by US ARO grant W91 INF-15-1-0479. 

L. Rey Vega is with the Departments of Electronics (FIUBA) and CSC-CONICET. Buenos Aires, Argentina (e-mail: 
lrey@fi.uba.ar, cgalar@fi.uba.ar). 

P. Piantanida is with the Laboratoire des Signaux et Systemes (L2S), CentraleSupelec, 91192 Gif-sur-Yvette. France (e-mail: 
pablo.piantanida@supelec.fr). 

Alfred O. Hero III is with the Department of Electrical Eng. & CompSci University of Michigan. Ann Arbor, MI, USA 
(e-mail: hero@umich.edu). 


January 19. 2016 


DRAFT 



2 


I. Introduction 


A. Motivation and related works 

Distributed source coding is an important branch of study in information theory with enormous 
relevance for the present and future technology. Efficient distributed data compression may be 
the only way to guarantee acceptable levels of performance when energy and link bandwidth 
are severely limited as in many real world sensor networks. The distributed data collected by 
different nodes in a network can be highly correlated and this correlation can be exploited at 
the application layer, e.g., for target localization and tracking or anomaly detection. In such 
cases cooperative joint data-compression can achieve a better overall rate-distortion trade-off 
that independent compression at each node. 

Complete answers to the optimal trade-offs between rate and distortion for distributed source 
coding are scarce and the solution to many problems remain elusive. Two of the most important 
results in information theory, Slepian-Wolf solution to the distributed lossless source coding 
problem [TJ and Wyner-Ziv [;2j single letter solution for the rate-distortion region when side 
information is available at the decoder provided the kick-off for the study of these important 
problems. Berger and Tung [|3), [|4j generalized the Slepian-Wolf problem when lossy recon¬ 
structions are required at the decoder. It was shown that the region obtained, although not tight 
in general, is the optimal one in several special cases 0-0 and strictly suboptimal in others 
Q. Heegard and Berger |T0] considered the Wyner-Ziv problem when the side information at 
the decoder may be absent or when there are two decoders with degraded side information. 
Timo et al |TT] correctly extended the achievable region for many (> 2) decoders. In [12] and 
the references therein, the complementary delivery problem (closely related to the Heegard- 
Berger problem) is also studied. The use of interaction in a multiterminal source coding setting 
has not been so extensively studied as the problems mentioned above. Through the use of 
multiple rounds of interactive exchanges of information explicit cooperation can take place using 
distributed/successive refinement source coding. Transmitting “reduced pieces” of information, 
and constructing an explicit sequential cooperative exchange of information, can be more efficient 
that transmitting the “total information” in one-shot. 

The value of interaction for source coding problems was first recognized by Kaspi in his sem¬ 


inal work [13|, where the interactive two-terminal lossy source coding problem was introduced 
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and solved under the assumption of a finite number of communication rounds. In [ 14] it is shown 
that interaction strictly outperforms (in term of sum rate) the Wyner-Ziv rate function. There 


are also several extensions to the original Kaspi problem. In [15] the interactive source coding 
problem with a helper is solved when the sources satisfy a certain Markov chain property. In 
|[T6|-[ 18|] other interesting cases where interactive cooperation can be beneficial are studied. To 
the best of our knowledge, a proper generalization of this setting to interactive multiterminal 
(> 2) lossy source coding has not yet been observed. 


B. Main contributions 

In this paper, we consider the three-terminal interactive lossy source coding problem presented 
in Fig. [T] We have a network composed of 3 nodes which can interact through a broadcast rate- 
limited -error free- channel. Each node measures the realization of a discrete memoryless source 
(DMS) and is required to reconstruct the sources from the other terminals with a fidelity criterion. 
Nodes are allowed to interact by interchanging descriptions of their observed sources realizations 
over a finite number of communication rounds. After the information exchange phase is over, 
the nodes try to reconstruct the realization of the sources at the other nodes using the recovered 
descriptions. 

The general rate-distortion region seems to pose a formidable mathematical problem which 
encompass several known open problems. However, several properties of this problem are 
established in this paper. 


General achievable region 

We derive a general achievable region by assuming a finite number of rounds. This region 


is not a trivial extension of Kaspi’s region [13] and the main ideas behind its derivation are 
the exchange of common and private descriptions between the nodes in the network in order to 
exploit optimality the different side informations at the different nodes. As in the original Kaspi’s 
formulation, the key to obtaining the achievable region is the natural cooperation between the 
nodes induced by the generation of new descriptions based on the past exchanged description. 
However, in comparison to Kaspi’s 2 node case, the 3 nodes interactions make significant 
differences in the optimal action of each node at the encoding and decoding procedure in 
a given round. At each encoding stage, each node need to communicate to two nodes with 
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Figure 1: Three-Terminal Interactive Source Coding. There is a single noiseless rate-limited 
broadcast channel from each terminal to the other two terminals. Dij denotes the average per- 
letter distortion between the source and X™ measured at the node i for each pair i ^ j. 


different side information. This is reminiscent of the Heegard-Berger problem [ |1Q| , [lfj, whose 
complete solution is not known, when the side information at the decoders is not degraded. 
Moreover, the situation is a bit more complex because of the presence of 3-way interaction. 
This similarity between the Heegard-Berger problem leads us to consider the generation of two 
sets of messages at each node: common messages destined to all nodes and private messages 
destined to some restricted sets of nodes. On the other hand, when each node is acting as a 
decoder the nodes need to recover a set of common and private messages generated at different 
nodes (i.e. at round l node 3, needs to recover the common descriptions generated at nodes 1 
and 2 and the private ones generated also at nodes 1 and 2). This is reminiscent of the Berger- 
Tung problem |4]-[6|, [19] which is also an open problem. Again, the situation is more involved 
because of the cooperation induced by the multiple rounds of exchanged information. Particularly 
important is the fact that, in the case of the common descriptions, there is cooperation based 
on the conditioning on the previous exchanged descriptions in addition to cooperation naturally 
induced by the encoding-decoding ordering imposed by the network. This explicit cooperation 
for the exchange of common messages is accomplished through the use of a special binning 
technique to be explained in Appendix [Bj 
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Besides the complexity of the achievable region, we give an inner bound to the rate-distortion 
region that allows us to recover the two node Kaspi’s region. We also recover several previous 
inner bounds and rate-distortion regions of some well-known cooperative and interactive -as 
well as non-interactive- lossy source coding problems. 


Special cases 

As the full problem seems to offer a formidable mathematical complexity, including several 
special cases which are known to be long-standing open problems, we cannot give a full converse 
proving the optimality of the general achievable region obtained. However, in Section [V] we 
provide a complete answer to the rate-distortion regions of several specific cooperative and 
interactive source coding problems: 

(1) Two encoders and one decoder subject to lossy/lossless reconstruction constraints without 
side information (see Fig. [2]). 

(2) Two encoders and three decoders subject to lossless/lossy reconstruction constraints with 
side information (see Fig. [3]). 

(3) Two encoders and three decoders subject to lossless/lossy reconstruction constraints, reversal 
delivery and side information (see Fig. [4]). 

(4) Two encoders and three decoders subject to lossy reconstruction constraints with degraded 
side information (see Fig. [5]). 

(5) Three encoders and three decoders subject to lossless/lossy reconstruction constraints with 
degraded side information (see Fig. [6]). 

Interestingly enough, we show that for the two last problems, interaction through multiple 
rounds could be helpful. Whereas for the other three cases, it is shown that a single round 
of cooperatively exchanged descriptions suffices to achieve optimality. Table [j] summarizes the 
characteristics of each of the above mentioned cases. 

Next we summarize the contents of the paper. In Section [II] we formulate the general problem. 
In Section [TIT] we present and discuss the inner bound of the general problem. In Section [TV] we 


show how our inner bound contains several results previously obtained in the past. In Section 
|V| we present the converse results and their tightness with respect to the inner bound for the 


special cases mentioned above providing the optimal characterization for them. In Section VI 


we present a discussion of the obtained results and their limitations and some numerical results 
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Cases 

Ri 

R.2 

Rs 

Constraints at Node 1 

Constraints at Node 2 

Constraints at Node 3 

( 1 ) 


7^0 

= 0 

0 (is not reconstructing 

any source) 

0 (is not reconstructing 

any source) 

Pr (X?, ± Xi 1 ) < 6 

E [d(X&,XJ)] < D 32 

( 2 ) 


7^0 

= 0 

E [<*(*&, X?)] < D 12 

Pr (Xji ^ X?) < e 

Pr (X 3 "i ^ Xp) < £ 

E [d(X 3 n 2 ,X 2 "j] < D 32 

(3) 

7^0 

^0 

= 0 

E [d(Ar 2 ,AT)] < D 12 

Pr (xr, ± xr) < t 

Pr (x 3 " 2 ^ X 2 ") < £ 
e [d(xr l 5 xr)] < d 3 i 

(4) 

^0 

7^0 

= 0 

E [d(Xr 2 ,X?)] < Di 2 

e [d(xy 1 ; xr)] <d 21 

E 

E 

d(xr l5 x r) 
d(xr 2 ,xr)' 

< D 3 i 

< d 32 

(5) 

^0 

7^0 

^0 

0 (is not decoding) 

Pr (x 2 "i ^ xr) < e 

E [d(X 2 " 3 ,X 3 n )] < D 23 

Pr (Xjr ± Xr) < £ 

E [d(X 3 n 2 ,Xr)] < D 32 


Table I: Special cases fully characterized in Section 


V 


concerning the new optimal cases from the previous Section. Finally in Section VII we provide 
some conclusions. The major mathematical details are relegated to the appendixes. 

Notation: We summarize the notation. With x n and upper-case letters X n we denote vectors 
and random vectors of n components, respectively. The ?'-th component of vector x n is denoted 
as Xi. All alphabets are assumed to be finite. 

Entropy is denoted by //(•) and mutual information by /(•;■). H 2 {p) denotes the entropy 
associated with a Bernoulli random variable with parameter p. With h(-) we denote differential 
entropy. Let X, Y and V be three random variables on some alphabets with probability distribu¬ 
tion pxyv■ When clear from context we will simple denote px {'■<') with p(x). If the probability 
distribution of random variables X,Y,V satisfies p(x\yv) = p(x\y) for each x,y,v, then they 
form a Markov chain, which is denoted by X -e- Y -e- V. 

The probability of an event A is denoted by Pr {A}, where the measure used to compute it 
will be understood from the context. Conditional probability of a set A with respect to a set B 
is denoted as Pr |*4|£>}. The set of strong typical sequences associated with random variable X 
(see appendix |Aj) is denoted by 7p^ e , where e > 0. We simply denote these sets as 77' when 
clear from the context. The cardinal of set A is denoted by ||A||. The complement of a set is 
denoted by A. With Z> 0 and R> ;3 we denote the integers and reals numbers greater than a and 
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ft respectively, co {*4.} denotes the convex hull of a set A £ M , where N e N. 


II. Problem formulation 

Assume three discrete memoryless sources (DMS’s) with alphabets and pmfs given by (A \ x 
X 2 x X 3 ,p Xl x 2 x 3 ) and arbitrary bounded distortion measures: dj : Xj x X 3 — * E> 0 , j £ 
Af = {1,2,3} where {X 3 } 3eM are finite reconstruction alphabet^] We consider the problem 
of characterizing the rate-distortion region of the interactive source coding scenario described 
in Fig. [T] In this setting, through K rounds of information exchange between the nodes each 
one of them will attempt to recover a lossy description of the sources that the others nodes 
observe, e.g., node 1 must reconstruct -while satisfying distortion constraints- the realization 
of the sources X% and AT,' observed by nodes 2 and 3. Indeed, this setting can be seen as a 


generalization of the well-known Kaspi’s problem [131. 


Definition 1 (K-step interactive source code): A A'-step interactive //-length source code, de¬ 
noted for the network model in Fig. |TJ is defined by a sequence of encoder mappings: 


X? X xj^x- 

■ ■ X Jt 1 X Jt l ) Jl , 

(1) 

X.? x {Jl x J 3 1 x • 

1 

X 

1 

X 

(2) 

X. X {Jl X Jl X ■ 

K 

T 

K 

X 

K 

X 

(3) 


with l £ [1 : A'] and message sets: J\ = {1, 2,... ,X-} , X[ £ Z> 0 , i £ Af, and reconstruction 
mappings: 

9ij ■ L” X (8) (X x " ' x X) j■ (4) 

mG Ai, rriy^i 

The average per-letter distortion and the corresponding distortion levels achieved at the node i 
with respect to source j are: 


E 


dj(x?,x$) <D tj i,j £ M, 


with 


d (x n , y n ) = - V d (%m, Vm) 
n 


(5) 


( 6 ) 


m= 1 


'The problem can be easily generalized to the case in which there are different reconstruction alphabets at the terminals. It 
can also be shown that all the results are valid if we employ arbitrary bounded joint distortion functions, e.g. at node 1 we use 

d(X 2 ,X 3 -X 2 ,X 3). 
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In compact form we denote a A'-step interactive source coding by (n, K , A, Q) where T and Q 
denote the sets of encoders and decoders mappings. 

Remark 1: The code definition depends on the node ordering in the encoding procedure. 
Above we defined the encoding functions {/{, f! 2 , ./y}, assuming that in each round node 1 
acts first, followed by node 2, and finally by node 3, and the process beginning again at node 1. 

Definition 2 (Achievability and rate-distortion region): Consider R = ( 1! \. R> ■ As) and D = 
(D 12 ,D 13 ,D 21 ,D 23 ,D 31 ,D 32 ). The rate vector R is (D, A')-achievable if Ve > 0 there is 
nfie, K) E N such that Vn > n 0 (e, A') there exists a A'-step interactive source code (n, K, A, Q) 
with rates satisfying: 

1 K 

- V log Ill'll < Ri + e, ieM (7) 

71 ' 

1=1 

and with average per-letter distortions at node i with respect to source j: 


E 


dfiX^X") <D ij + e, i,jeM, , 


( 8 ) 


where Xg = g tj , (g) 
TZ 3 (D, K) is defined by: 


meM, m^i X ■ • • x jfi') J , i j E M. The rate-distortion region 


77 -3 (D, A') = |r : R is (D, A')-achievable| (9) 

Similarly, the D-achievable region TZ 3 (D) is given by 77 3 (D ) = , A : >(D, A'jQ that is: 

77.3(D) = |r : R is (D, AT)-achievable for some K E Z>i j . ( 10 ) 


Remark 2: By definition 77-3(D, AT) is closed and using a time-sharing argument it is easy to 
show that it is also convex VA' eZy. 

Remark 3: 77. 3 (D, K) depends on the node ordering in the encoding procedure. Above we 
defined the encoding functions { f\ , f 2 , / 3 } [1, assuming that in each round node 1 acts first, 
followed by node 2, and finally by node 3, and the process beginning again at node 1. In this 
paper we restrict the analysis to the canonical ordering (1 —* 2 —>• 3). However, there are 3! = 6 
different orderings that generally lead to different regions and the (D, A")-achievable region 


2 Notice that this limit exists because it is the union of a monotone increasing sequence of sets. 
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defined above is more explicitly denoted TZ 3 (D, K, a c ), where a r is the trivial permutation for 
Ai. The correct (D, K )-achievable region is: 

n 3 (B } K)= U ^ 3 (d,a» ( 11 ) 

where E(Ai) contains all the permutations of set Ai. The theory presented in this paper for 
determining 1Z 3 (D,K,a c ) can be used on the other permutations a ^ a c to compute (fTT[p| 

III. Inner Bound on the Rate-Distortion Region 
In this Section, we provide a general achievable rate-region on the rate-distortion region. 

A. Inner bound 

We first present a general achievable rate-region where each node at a given round l will 
generate descriptions destined to the other nodes based on the realization of its own source, the 
past descriptions generated by a particular node and the descriptions generated at the other nodes 
and recovered by the node up to the present round. In order to precisely describe the complex 
rate-region, we need to introduce some definitions. For a set A, let C (.4) = 2 A \ {A, 0} be the 
set of all subsets of A minus A and the empty set. Denote the auxiliary random variables: 

Ui-+ S ,i, S eC (Ai) , i <f S, l — 1,... ,K. (12) 

Auxiliary random variables {U^s.i} will be used to denote the descriptions generated at node i 
and at round l and destined to a set of nodes S £ C (Ai) with i £ S. For example, Ui^. 2 3 ,i denote 
the description generated at node 1 and at round l and destined to nodes 2 and 3. Similarly, 
will be used to denote the descriptions generated at node 1 at round l and destined 
only to node 2. We define variables: 

W[jj] = Common informatioij^] shared by the three nodes available at node i at round l before 
encoding 


Tt should be mentioned that this is not the most general setting of the problem. The most general encoding procedure will 
follow from the definition of the transmission order by a sequence ti, £2, G,..., t\\M\\xK with U £ M. This will cover even the 
situation in which the order can be changed in each round. To keep the mathematical presentation simpler we will not consider 
this more general setting. 


4 Not to be confused with the Wyner’s definition of common information 


20 


January 19, 2016 


DRAFT 



10 


V[s,/,i] = Private information shared by nodes in S' G C (A4) available at node i G S', at round 
l, before encoding 

In precise terms, the quantities introduced above for our problem are defined by: 

W[l,l]={Ul^23,k, £4—>13,fc, s-12, 

W[2,Z]=W[1 ) Z] U ^i_23,i > 

>V[3,Z]=VV ; [2 ) Z] , 

V[l 2 ,l,l]={Ul^ 2 ,k, U 2 ^l,k} l k = 1 , V[ 12 ,Z, 2 ] = V[i 2 ,i,l] u t/l_». 2 ,i , 

V[l3,Z,l]={t/l_).3,fe, ) ^[!3,i,3] — V[13,Z,1] U U\^,l , 

V[23,Z,2] = {f^2-s>3,A;) ^3-J2,fc}fc = i j V[23,Z,3] = V[23,Z,2] U t/2^3,Z • 

Before presenting the general inner bound, we provide the basic idea of the random coding 

scheme that achieves the rate-region in Theorem [T] for the case of K communication rounds. 

Assume that all codebooks are randomly generated and known to all the nodes before the 
information exchange begins and consider the encoding ordering given by 1 —» 2 —>• 3 so that 
we begin at round l = 1 in node 1. Also, and in order to maintain the explanation simple and 
to help the reader to grasp the essentials of the coding scheme employed, we will consider that 
all terminal are able to recover the descriptions generated at other nodes (which will be the 
case under the conditions in our Theorem [TJ. From the observation of the source X”, node 1 
generates a set of descriptions for each of the other nodes connected to it. In particular it generates 
a common description to be recovered at nodes 2 and 3 in addition to two private descriptions for 
node 2 and 3, respectively, generated from a conditional codebook given the common description. 
Then, node 2 tries to recover the descriptions destined to it (the common description generated 
at 1 and its corresponding private description), using XV; as side information, and generates its 
own descriptions, based on source XV; and the recovered descriptions from node 1. Again, it 
generates a common description for nodes 1 and 3, a private description for node 3 and another 
one for node 1. The same process goes on until node 3, which tries to recover jointly the common 
descriptions generated by node 1 and node 2, and then the private descriptions destined to him 
by node 1 and 2. Then generates its own descriptions (common and private ones) destined to 
nodes 1 and 2. Finally, node 1 tries to recover all the descriptions destined to it generated by 
nodes 2 and 3 in the same way as previously done by node 3. After this, round l — 1 is over, 
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and round l = 2 begins with node 1 generating new descriptions using X™, its encoding history 
(from previous round) and the recovered descriptions from the other nodes. 

The process continues in a similar manner until we reach round l = K where node 3 recovers 
the descriptions from the other nodes and generates its own ones. Node 1 recovers the last 
descriptions destined to it from nodes 2 and 3 but does not generate new ones. The same holds 
for node 2 who only recovers the descriptions generated by node 3 and thus terminating the 
information exchange procedure. Notice that at the end of round K the decoding in node 1 and 
node 2 can be done simultaneously. This is due to the fact that node 1 is not generating a new 
description destined to node 2. However, in order to simplify the analysis and notation in the 
appendix we will consider that the last decoding of node 2 occurs in round K + After all 
the exchanges are done, each node recovers an estimate of the other nodes, source realizations 
by using all the available recovered descriptions from the K previous rounds. 

Theorem 1 (Inner bound): Let Tt 3 (D,K) be the closure of set of all rate tuples satisfying: 


K 

Rl=Y) (*li23 + flfL + Rts) <13) 

1 = 1 
K 

Ri = J2 (^£.13 + i + M'L) (M) 

i=i 

K 

R 3 = Y) (R^ + (15) 

1=1 
K 

r 1 + r 2 = Y) {Rl \23 + R 2 l 13 + Rils + R %\3 + Rl \2 + 4‘i 1 ) (16) 

1=1 
K+l 

TD 1 TD _ \ A ( p(0 _1_ rjO - 1) 1 p(0 1 1 p(0 1 p(0 1 

Ki + K 3 - (-23 + -^3^12 + K 1^2 + K 3^2 + K 1^3 + -^3^1) l 1 '> 

1=1 
K 

R 2 + R 3 = Y j {RSI 13 + R 3 —i 12 + Rt 1 + R 3 l 1 + + R 3 l 3 ) (18) 

1=1 


5 This is clearly a fictitious round, in the sense that there is not descriptions generation on it. In this way, there is not 
modification of the final rates achieved by the procedure described if we consider this additional round. 
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where^jfor each Z e [1 : K]: 


p 1 
-/ L 


(0 

1—>23 


>I[X 1 -Ui 


1—>23,Z 


D 
-/ T- 


p 1 
i 1 


?4 13 >/(X 2 ;f/ 2 ^i3,z 

(0 


3—>12 


>/ X 3 ;f/ 3 ^i2 


^2W[l j /]V[l2,Z,l]V[2 3 ,/-l,3] 

-^ 3 VV[2,f] V[l 3 ,Z,l] V[2 3 ,Z,2] 

^iW[ 3i qV[i 2> i ) 2]V[i 3l , )3 ] 


i? 


i?. 


+ 44l3 > I (^XiX 2 ] Z/l—>23,/Z^2—>13,Z -^W^]V[i 3; / 5 i]V[2 3 ,/,2] 


(0 

1- >23 
(0 

2— >13 1“ - 1 L 3—>12 


+ 44 12 >/(X 2 X 3 ;[/ ; 


U, 


y 2—>13,/'-' 3—>12,Z 


-^lVV[2,i]V[l2,i,2] V[l 3j f j3 ] 


4423 + 44 12 > I {X\X^Ui^23,lU^V2,l-l V[i2,Z,l] V[23,Z-1,3]^ 




(z—1) 

3—>2 


>1 x 3] u, 


3—>2,Z—1 


^2W[2,Z]V[2 3 ,Z— 1,3]V[12,Z,2] 


d(0 

- n T->2 

d( 0 I pd — i) 
- fl l->2 “r rl 3->2 


> I (4fi; Z7l_>2,Z ^2VV[2,Z]V[23,Z,2]V[12,Z,1] 

> I (4iX 3 ; Z/l->.2,Z^3->12,Z-l ^2'H ; [2,Z]V[23,Z~1,3]V[12,Z,1] 


i? 


(0 

1—>3 




1—>3,Z 


i? 


(0 


-^3VV[ 3 ^] V[23,i,3] V[1 3 ^,1] 

-^ 3 VV[ 3i Z]V[23,Z,2]V[i 3j z i3 ] 


l, 2—>3 > I [X 2 ] U 2 ^3,l 

44s + 44s > / (* 1 * 2 ; £^3,Z^3,Z -^ 3 VV[ 3 ,z]V[23,Z,2]V[i 3i z,i]^ 

44i > I (* 2 ; U 2 ^i X 1 W [1 ,z +1] V[i 2 ,z, 2 ]V[i 3 ,z +U ]) 

^lW [ i,z + i]V[i2,Z + l,l]V[l 3 ,Z )3 ] 


44l > I [X ? -U;^ hl 

44 i + 44 i > / (^2X3; U^U^i IlW[l,l + l]V[12,Z,2] V[i 3) z i3 ]^ 
with 44 s = 444 — 0 and Ui^sfl = ZT^ss'+i = 0 for S' G C (M) and i ^ S. 

With these definitions the rate-distortion region satisfied 

u n 3 (D,K)cn 3 (D,K), 

peT(D,K) 


(19) 

( 20 ) 
( 21 ) 
( 22 ) 

(23) 

(24) 

(25) 

(26) 

(27) 

(28) 

(29) 

(30) 

(31) 

(32) 

(33) 


(34) 


6 Notice that these definitions are motivated by the fact that at round 1, node 2 only recovers the descriptions generated by 
node 1 and at round K + 1 only recovers what node 3 already generated at round K. 

7 It is straightforward to show that the LHS of equation 
needed. 




34b is convex, which implies that the convex hull operation is not 
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where V(D,K) denotes the set of all joint probability measures associated with the following 
Markov chains for every l e [1 : K\: 

1) U 1^23,1 (A"i, yv^ji^]) -e- (A 2 , A 3 , V[i 2 ,i,l], V[13,/,1] J V[23,i,2]) ! 

2) Ui->2,1 (Ay, W[2,i], V[12,t,l]) -e_ (A 2 , A 3 , Vp^p], V[23,Z,2]) » 

3) -e- (-^ 1 , W[ 2 ,i], V[l 3 ; i,l]) -©- (A 2 , A 3 , V[ 12 ,J, 2 ], V[23,i,2]) 7 

4 ) U 2 ^l 3 ,l -e ~ (X 2 ,W [2 , l] ) -e- (^!, -^3, V[i 2 ,i, 2 ], V[13^ 5 3], V[23,i,2]) 7 

5) U 2 ^u -©- (A 2 , VV[3^], V[ 12 ,z, 2 ] ) (ACi, AC 3 , V[ 13 ^ ; 3 ], V[23,f,2]) 7 

6) */ 2 ->3,Z _e ~ (X 2 , W[ 3) j], V[23,i,2]) (Ay, A 3 , V[i 2 ,i+l,l], V[i 3 5 / 5 3 ] ) , 

7 ) U W i -e- (A 3 ,W[ 3) ,]) -©- (Ai, A 2 , V[i 2> j[+i,i], 12^3^3]) , 

8 ) U3—ti,i -e- (A 3 ,W w+ i],V[i 3 ,z, 3 ]) (Ai, A 2 , V[i 2 ,j+i,i], V[ 23 ,i, 3 ]) 7 

9) Uz^.2,1 (A 3 , VV[i, i+ i], V[ 2 3 ,z j3] ) ~ e ~ l(Ai, A 2 , V[i 2; ^_(_i 5 i],”V[ i3,i+i,i]) 7 

and such that there exist reconstruction mappings: 


9ij (Aj, V[ij t K+i,i\ A ij (35) 

with E dj(Xj,Xij ) < Dij for each i, j e Ai and i ^ j. 

The proof of this theorem is relegated to Appendix [C] and relies on the auxiliary results 
presented in Appendix [A] and the theorem on the cooperative Berger-Tung problem with side 
information presented in Appendix [Bj 

Remark 4: It is worth mentioning here that our coding scheme is constrained to use successive 
decoding , i.e., by recovering first the coding layer of common descriptions and then coding 
layer of private descriptions (at each coding layer each node employ joint-decoding). Obviously, 
this is a sub-optimum procedure since the best scheme would be to use joint decoding where 
both common and private informations can be jointly recovered. However, the analysis of this 
scheme is much more involved. The associated achievable rate region involves a large number of 
equations that combine rates belonging to private and common messages from different nodes. 
Also, several mutual information terms in each of these rate equations cannot be combined, 
leading to a proliferation of many equations that offer little insight to the problem. 

Remark 5: The idea behind our derivation of the achievable region can be extended to any 
number M (> 3) of nodes in the network. This can be accomplished by generating a greater 
number of superimposed coding layers. First a layer of codes that generates descriptions destined 


January 19, 2016 


DRAFT 




14 


to be decoded by all nodes. The next layer corresponding to all subsets of size M — 1, etc, until 
we reach the final layer composed by codes that generate private descriptions for each of nodes. 
Again, successive decoding is used at the nodes to recover the descriptions in these layers 
destined to them. Of course, the number of required descriptions will increase with the number 
of nodes as well as the obtained rate-distortion region. 

Remark 6: It is interesting to compare the main ideas of our scheme with those of Kaspi 


[13[. The main idea in [131 is to have a single coding tree shared by the two nodes. Each leaf 
in the coding tree is codeword generated either at node 1 or 2. At a given round each node 
knows (assuming no errors at the encoding and decoding procedures) the path followed in the 
tree. For example, at round /, node 1, using the knowledge of the path until round l and its 
source realization generate a leaf (from a set of possible ones) using joint typicality encoding 
and binning. Node 2, using the same path known at node 1 and its source realization, uses joint 
typicality decoding to estimate the leaf generate at node 1. If there is no error at these encoding 
and decoding steps, the previous path is updated with the new leaf and both -node 1 and 2- 
know the updated path. Node 2 repeats the procedure. This is done until round K where the 
final path is known at both nodes and used to reconstruct the desired sources. 

In the case of three nodes the situation is more involved. At a given round, the encoder at an 
arbitrary node is seeing two decoders with different side informatiorj^J In order to simplify the 
explanation consider that we are at round l in the encoder 1, and that the listening nodes are 
nodes 2 and 3. This situation forces node 1 to encode two sets of descriptions: one common 
for the other two nodes and a set of private ones associated with each of the listening nodes 2 
and 3. Following the ideas of Kaspi, it is then natural to consider three different coding trees 
followed by node 1. One coding tree has leaves that are the common descriptions generated 
and shared by all the nodes in the network. The second tree is composed by leaves that are 
the private descriptions generated and shared with node 2. The third tree is composed by leaves 
that are the private descriptions generated and shared with node 3. As the private descriptions 
refine the common ones, depending on the quality of the side information of the node that is the 
intended recipient, it is clear that descriptions are correlated. For example, the private description 
destined to node 2, should depend not only on the past private descriptions generated and shared 


^Because at each node the source realizations are different, and the recovered previous descriptions can also be different. 
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by nodes 1 and 2, but also on the common descriptions generated at all previous rounds in all 
the nodes and on the common description generated at the present round in node 1. Something 
similar happens for the private description destined to node 3. It is clear that as the common 
descriptions are to be recovered by all the nodes in the network, they can only be conditioned 
with respect to the past common descriptions generated at previous rounds and with respect to 
the common descriptions generated at the present round by a node who acted before (i.e. at 
round l node 1 acts before node 2). The private descriptions, as they are only required to be 
recovered at some set of nodes, can be generated conditioned on the past exchanged common 
descriptions and the past private descriptions generated and recovered in the corresponding set 
of nodes (i.e., the private descriptions exchanged between nodes 1 and 2 at round l, can only 
be generated conditioned on the past common descriptions generated at nodes 1, 2 and 3 and 
on the past private descriptions exchanged only between 1 and 2). 

We can see clearly that there are basically four paths to be cooperatively followed in the 
network: 

• One path of common descriptions shared by nodes 1, 2 and 3. 

• One path of private descriptions shared by nodes 1 and 2. 

• One path of private descriptions shared by nodes 1 and 3. 

• One path of private descriptions shared by nodes 2 and 3. 

It is also clear that each node only follows three of these paths simultaneously. The exchange of 

common descriptions deserves special mention. Consider the case at round / in node 3. This node 
needs to recover the common descriptions generated at nodes 1 and 2. But at the moment node 
2 generated its own common description, it also recovered the common one generated at node 1. 
This allows for a natural explicit cooperation between nodes 1 and 2 in order to help node 3 to 
recover both descriptions. Clearly, this is not the case for private descriptions from nodes 1 and 2 
to be recovered at node 3. Node 2 does not recover the private description from node 1 to 3 and 
cannot generate an explicit collaboration to help node 3 to recover both private descriptions. Note, 
however, that as both private descriptions will be dependent on previous common descriptions 
an implicit collaboration (intrinsic to the code generation) is also in force. In appendix [B] we 
consider the problem (not in the interactive setting) of generating the explicit cooperation for 
the common descriptions through the use of what we call a super-binning procedure, in order 
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to use the results for our interactive three-node problem. 


IV. Known Cases and Related Work 

Several inner bounds and rate-distortion regions on multiterminal source coding problems can 
be derived by specializing the inner bound p4| ). Below we summarize only a few of them. 

1) Distributed source coding with side information [ : 4f [19]: Consider the distributed source 
coding problem where two nodes encode separately sources X[ and Xf to rates (R\, R 2 ) and a 
decoder by using side information X” must reconstruct both sources with average distortion less 
than D\ and D 2 , respectively. By considering only one-round/one-way information exchange 
from nodes 1 and 2 (the encoders) to node 3 (the decoder), the results in 0, © can be 


recovered as a special case of the inner bound (|34|). Specifically, we set: 


Ui^ 2 3 : l=U 2 ^i3 : i = C 3 _).12,; = Ci_).2,Z = U 2 ^1 } 1 = = U3^, 2 J = 0, VZ 

Ui^3j=U 2 ^3j = 0, V/ > 1 . 

In this case, the Markov chains of Theorem [7] reduce to: 


£4— >3, i-s-Xi -e- (X 2 , X 3 , C/2^3, 1) 5 
U 2 ^, 3 ,i~ e ~X 2 -e- (X 1 ,X 3 ,U 1 ^ 1 ) , 

and thus the inner bound from Theorem [I] recovers the results in [ [T9 1 


(36) 

(37) 


R 1 >I(X 1 -U 1 ^3,i\X 3 U 2 ^3,i) , 

(38) 

R 2 >I{X 2 -U 2 ^3AX3U^3,l) , 

(39) 

Ri + 7?2>7(X 1 X 2 ; C/ 1 _ 3>1 C/2-, 3)1 |X 3 ) . 

(40) 


2) Source coding with side information at 2-decoders m m Consider the setting where 
one encoder X\ l transmits descriptions to two decoders with different side informations (X[. X") 
and distortion requirements D 2 and I).>. Again we consider only one way/round information 
exchange from node 1 (the encoder) to nodes 2 and 3 (the decoders). 

In this case, we set: 

U 2 -H3,l=U 3 _n 2} i = U 2 ^IJ = = f7 3 _^ 2 ,Z = U 2 ^3 t i = 0, V/ 

ul^ 2 3,l—Ul-^ 2 3,l = i'l->2,l = U^3,l = 0, VZ > 1 . 
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The above Markov chains imply 

{U\-+ 23,1 , Ui-^2,1: £^1—>3,1 )^X 1 -^-(X 2 ,X 3 ) (41) 

and thus the inner bound from Theorem [T] reduces to the results in [ |T0] |, [11] 

#i>max {/(X i; f/^ 2 3,i|X 2 ), J(X i; f/i^ 23 ,i|X 3 )} 

+/(Xi; f/^lAWi^i) + /(X i; C/^i 1X3^23,0 . (42) 


5) Two terminal interactive source coding [13]: Our inner bound ( [34] ) is basically the gener¬ 
alization of the two terminal problem to the three-terminal setting. Assume only two encoders- 
decoders X\ l and X] which must reconstruct the other terminal source 3 with distortion con¬ 
straints D i and D 2 , and after K rounds of information exchange. Let us set: 

Ul^23,l=U 2 ^13,l = 6^3—>12,2 = (A->3,2 = (^3—>1,1 = (^2->3,2 = ^3->2,l = 0, VI 

X 3 =0 . 

The Markov chains become 


U 1 -> 2 ,i-e-(X 1 ,V l i2,i,ii)-e-X 2 , (43) 

2 —>i,i (X 2 , V[i 2j / i2 ]) X 2 , (44) 

for l G [1 : K] and thus the inner bound from Theorem [I] permit us to obtain the results in [ 13 ] 

Ri>I(X r ,V [12 , K+lA] \X 2 ) , (45) 

R 2 >I(X 2 ; V[i2,a-+i, 2 ] |^i) • (46) 

4) Two terminal interactive source coding with a helper [15]: Consider now two encoders/decoders, 
namely X% and X", that must reconstruct the other terminal source with distortion constraints 
D 2 and D 3 , respectively, using K communication rounds. Assume also that another encoder 
X r [ provides both nodes (2, 3) with a common description before beginning the information 
exchange and then remains silent. Such common description can be exploited as coded side 
information. Let us set: 


(^2—>13,2 — (^3—>12,1 — 6^1— >3,1 — 6^1— >2,1 — (A—>3,1 — t^2->l,Z — C^3—>1,1 — 0 , VZ 

Ui^. 23 j = 0, VI > 1 . 
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The Markov chains reduce to: 

Ul-> 23,1 -e-Xx-e-p^Xa) , (47) 

U 2 ^3,l -e_ (X 2 , (7l->23,l) V[23,i,2]) -e_ (Xi, X 3 ) , (48) 

U;i^2,l “©“(^3) ^1^23,1) V[23,i, 3 ]) -©- (^l, X 2 ) . (49) 


An inner bound to the rate-distortion region for this problem reduces to (using the rate equations 
in our Theorem [Tj) 

Ri> max {/(X i; t/i^ 2 3 ,i|X 2 ), /(X i; (7^23 ,i|X 3 )} , (50) 

R 2 >I(X 2 ; V[23,a-+i,2] 1X3(7^23,1) , (51) 

R 3 >I(X 3 ; V[23,a-+i,2] |X2(7^23,i) • (52) 


This region contains as a special case the region in 151. In that paper it is further assumed 
(in order to have a converse result) that Xi -©- A7 -e- X 2 . Then, the value of R\ satisfies 
Ri > I(X\: 17 ^ 23,1 IX 2 ). Obviously, with the same extra Markov chain we obtain the same 
limiting value for Ri and the above region is the rate-distortion region. 


V. New Results on Interactive and Cooperative Source Coding 


A. Two encoders and one decoder subject to lossy/lossless reconstruction constraints without 
side information 

Consider now the problem described in Fig. [2] where encoder 1 wishes to communicate the 
source Xj to node 3 in a lossless manner while encoder 2 wishes to send a lossy description 
of the source XJj to node 3 with distortion constraint Z7 31 . To achieve this, the encoders use K 
communication rounds. This problem can be seen as the cooperating encoders version of the 
well-known Berger-Yeung [5J problem. 

Theorem 2: The rate-distortion region of the setting described in Fig. [8] is given by the union 
over all joint probability measures px 1 x 2 u 2 ^i 3 suc h that there exists a reconstruction mapping: 


<732 (Xi, (/ 2 -K 1 . 3 ) — X 32 with E d(X 2l X 32 ) 


< D32 


(53) 


January 19, 2016 


DRAFT 





19 



Figure 2: Two encoders and one decoder subject to lossy/lossless reconstruction constraints 
without side information. 


of the set of all tuples satisfying: 

Ri>H(X 1 \X 2 ) , 
R2>I(X 2 ;U 2 ^ 13 \X 1 ) , 

Ri + R2> H(X 1 ) + I(X 2 - U^slX,) 


(54) 

(55) 

(56) 


The auxiliary random variable U 2 ^i 3 has a cardinality bound of ||^Y 2 —^ 13 1| < ||A’ 1 || \\X 2 \\ + 1. 

Remark 7: It is worth emphasizing that the rate-distortion region in Theorem [2] outperforms the 
non-cooperative rate-distortion region first derived in [5 j. This is due to two facts: the conditional 


entropy given in the rate constraint ( |54| ) which is strictly smaller than the entropy H(X\) present 
in the rate-region in [5j, and the fact that the random description C/ 2^13 may be arbitrarily 
dependent on both sources (X\,X 2 ) which is not the case without cooperation [5]. Therefore, 
cooperation between encoders 1 and 2 reduces the rate needed to communicate the source X 1 
while increasing the optimization set of all admissible source descriptions. 

Remark 8: Notice that the rate-distortion region in Theorem [2] is achievable with a single 
round of interactions K = 1, which implies that multiple rounds do not improve the rate- 
distortion region in this case. This holds because of the fact that node 3 reconstruct X\ in a 
lossless fashion. 

Remark 9: Although in the considered setting of Fig. [ 8 ] node 1 is not supposed to decode 
neither a lossy description nor the complete source X7 l , if nodes 1 and 3 wish to recover the same 
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descriptions the optimal rate-region remains the same as given in Theorem [2j The only difference 
relies on the fact that node 1 is now able to find a function g 32 (Xi, U 2 ^\ 3 ) = X 12 which must 


satisfy an additional distortion constraint E 


< D 12 . In order to show this, it is 


d(X 2 ,X 12 ) 

enough to check that in the converse proof given below the specific choice of the auxiliary 
random variable already allows node 1 to recover a general function X 12 [ t ] = gi 2 (X^j, C/ 2 —>-i 3 [*]) 
for each time t G { 1 ,..., n}. 

Proof: The direct part of the proof simply follows by choosing: 

U 3 -H 2 ,l=Ui-+ 3 j = U^ 2 ,i = t/ 2 ->l,Z = ^2—>3,Z = ^3->l,Z = ^3-s>2,Z = 0, VZ 

Ui^ 23; i=X 1 , U^i = C/ 2 ^i3,Z = 0 V l > 1 , 

and thus the rate-distortion region ( [34] ) reduces to the desired region in Theorem [2] where for 
simplicity we dropped the round index. We now proceed to the proof of the converse. 

If a pair of rates (Ri, R 2 ) and distortion D 32 are admissible for the A'-steps interactive 
cooperative distributed source coding setting described in Fig. [ 8 j then for all e > 0 there exists 
7i 0 (e, AT), such that Vn > n 0 (e,K) there exists a A'-steps interactive source code (n, K.T .Q) 
with intermediate rates satisfying: 


1 


n 


K 


log \\J-\\ < Ri + £ , « e {1,2} 


(57) 


1=1 


and with average per-letter distortions with respect to the source 2 and perfect reconstruction 
with respect to the source 1 at node 3: 


d(X%,X ») 


E 


Pr X" f X. 


< D 32 + £ , 

< £ , 


where 


Y n — n I q\y'K\ 

^32 — ZZ32 l J 1 i J 2 


j ^31 — ^731 l J 1 5 <7 2 


For each t e {1,..., n}, define random variables t/ 2 ^i 3 [ t ] as follows: 

TT — f 'rd'-K} q-\y-.K\ Y Y 

<-A-s-13[t] — , <T 2 , 


(58) 

(59) 

(60) 

(61) 


By the condition ( p9| ) which says that Pr f Xf J < e and Fano’s inequality [211, we have 

H( A7|A' 3 ”) < Pr (vr + X 31 ) 108,(11^11 - 1) + H-i (pr(Af + A?,)) 4 
where e n (e) —> 0 provided that e —> 0 and n —> oo. 


ne r 


(62) 
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1) Rate at node 1: For the first rate, we have 

n{Ri + e) > H (jf* 1 ) (63) 

> (64) 

= ntf (X^X,) - H (X?\X (65) 
= nH{X 1 \X 2 ) - H (X™\X^jl 1]K] jt K] ) (66) 

> n//(X 1 |X 2 ) - H{X^X^) (67) 

( 6 ) 

> n77(Xi|X 2 ) - ne n , (68) 


where 


step (a) follows from the fact that by definition of the code the sequence j \ 1 'is a function 
of the source X£ and the vector of messages 

step (6) follows from the code assumption that guarantees the existence of a reconstruction 
function XV iX = g 31 f 


step (c) follows from Fano’s inequality in ([62]). 
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2) Rate at node 2: For the second rate, we have 

n(Ro +e)> H 


= i ( J^-XI ) + / ( jI vk \ x%\x{ 


> i [jt K] -,x?xz 


r[l :K]. 


(a) n 

> i (J2 :K] \ x i') +Y, i { j * :K] ' x w\ x m x m^ x m-i] x 2[i-.t-i] 


t =1 
n 


I (jf^X?) +J2 I (^ K] Xl [t+ l:n]X 1[1:t - 1] X 2 [ 1:t _ 1]; X 2[t] |X 

t= 1 

> / (jf^xr) + J]/ (C/ 2 ^ 13[t] ;X 2[t] |X 1[t] ) 

£=1 
n 

I (jf K| ; A?) + V / (C/^,3,0,; A 2|0| |A 1[0I , Q = () 

t=l 

= I (jf K| ; A?) +nl (U^ im -,X m \X m ,Q) 


m 


(c) 


(d) 


(/) 


> / (jf^X" )+nl{ C/ 2 ^ 13 ; X^Xy 


(9) 


> nl ( f/ 2 _s. 13 ; X 2 |Xi) , 


where 


(69) 

(70) 

(71) 

(72) 

(73) 

(74) 

(75) 

(76) 

(77) 

(78) 


step (a) follows from the chain rule for conditional mutual information and non-negativity 
of mutual information, 

step ( b ) follows from the memoryless property across time of the sources (X", X 2 ), 
step (c) follows from the non-negativity of mutual information and definitions ( [771] ), 
step ( d ) follows from the use of a time sharing random variable 0 uniformly distributed 
over the set (1,..., n}, 

step (e) follows from the definition of the conditional mutual information, 
step (/) follows by letting a new random variable f/ 2 ->i 3 = (U 2 ^i 3 \q],Q), 
step {(]) follows from the non-negativity of mutual information. 
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3) Sum-rate of nodes 1 and 2: For the sum-rate, we have 
n{R i + R 2 + 2e) > H ( j \ l:A] ) + n(R 2 + e) 


(a) 


> H ( j } hK] ) + / (XI )+nl( £7 2 _> 13 ; X 2 \X x 


r[l-.K]_ 


= Hi j} 1:K] \j} 1:K] 


+ I (Ji [1:A1 ; jt K] 


+ / (j\ l:K] - A7) + nl ( L7 2 _> 13 ; X 2 \X 1 
> I (j} l:K] ;X\ l \ji UK] 

+ / + nl ( C/ 2 ^ 13 ; X 2 |X 3 


+ / (Ji [1:A] ; jf* 1 


= I ( j} 1 :K] -x ?) + / ( X?Jl XMi -ft 


r n rr\\'K\. rr[\'K\ 


+ nl [U 2 ^ 13 ;X 2 \X 1 


(b) 

= n 


H(X 1 ) + I[ U 2 ^X 2 \X x 
H (x?\j} 1:K] 


(c) 

> n 

(d) 

> n 

(e) 

> n 


+ I[X?j} 1:K] -,ji 1:K] 


H{Xf) + I ( U 2 ^ 13 -X 2 \X ,) -Hi X?\jl 1:K] j£ :K] 


H{Xf) + I (tf 2 _> 13 ; X 2 \X^ - H(X\ l \Xf) 
H{Xf) +1 (u 2 ^i 3 ]X 2 \Xi 


(79) 

(80) 

( 81 ) 

(82) 

(83) 

(84) 

(85) 

( 86 ) 

(87) 


where 

• step (a) follows from inequality ( J77| ), 

• step ( b ) follows from the memoryless property across time of the source X'[\ 

• step (c) follows from non-negativity of mutual information, 

• step (d) follows from the code assumption that guarantees the existence of reconstruction 
function Xf = g 31 (jTj 1K \ J 2 ' K ^) and from the fact that unconditioning increases entropy, 


• step (e) from Fano’s inequality in ( |62| ). 

4) Distortion at node 3: Node 3 reconstructs lossless X™ x = g 31 J^' 1 ^ 

r[l:K] rj\l:K) 


and lossy 

X” 2 = g 32 For each t e {1,..., n}, define a function X 32 [t] as beging the t-th 

coordinate of this estimate: 


x m] (E/2-13K]) = JS2[<] (jX\j 1 K] ) . 


( 88 ) 
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The component-wise mean distortion thus verifies 

D 32 + e>E[d [x 2 ,g 31 (j! 1:K \jt K] )) 


- E d ( X 2 M ’ ^32 [t] {U 2 ^l3[t ]) 


t =1 


“E E ^ (^ 2 [Q], X 32[ Q] (U 2 ^ 13 [ Q] ) 


i=l 


d \ X 2 [Q], A" 3 2[Q] (U 2 ^ 13[Q] ) 

d(x 2 l X 32 (u 2 ^ 13 


Q — t 


= E 
= E 

where we defined function X 32 by 

X 32 = X 32 {Q,U 2 ^ 13[q] ) = X 32 [q] (C/ 2 —>i3[Q]) 

This concludes the proof of the converse and thus that of the theorem. 


(89) 

(90) 

(91) 

(92) 

(93) 

(94) 


B. Two encoders and three decoders subject to lossless/lossy reconstruction constraints with 
side information 

Consider now the problem described in Fig. [3] where encoder 1 wishes to communicate the 
lossless the source X™ to nodes 2 and 3 while encoder 2 wishes to send a lossy description of its 
source Xj to nodes 1 and 3 with distortion constraints D 32 and D 32 , respectively. In addition to 
this, the encoders overhead the communication using K communication rounds. This problem 
can be seen as a generalization of the settings previously investigated in [3], |5]. 

Theorem 3: The rate-distortion region of the setting described in Fig. [3] is given by the union 
over all joint probability measures Px 1 x 2 x 3 U 2 ^i 3 U 2^3 satisfying the Markov chain 


and such that there exists reconstruction mappings: 

932 (Xi, X 3 , C 2 —>i 3 > U 2 ^ 3 )=X 32 with E 
<712 (2Cl, U 2 ^ 13 )=X l2 with E 


^-X 3 

d(X 2 ,X 32 ) 
d{X 2 ,X 12 ) 


< D 32 , 

< Dl2 5 


(95) 


(96) 

(97) 
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Figure 3: Two encoders and three decoders subject to lossless/lossy reconstruction constraints 
with side information. 


of the set of all tuples satisfying: 

Ri>H{X 1 \X 2 ) , (98) 

R 2 > i 3 ; X 2 \X l ) + I(U 2 __ 3 ; XfU^X^) , (99) 

Ri + R2> H(X 1 \X 3 ) + I(U 2 ^ 13 U 2 ^ X 2 \X 1 X 3 ). (100) 

The auxiliary random variables have cardinality bounds: ||W 2 ->i 3 || < ||Afi||||Af 2 || + 2, j|W 2 ->. 3 || < 

Kiiraiit^di + i. 

Remark 10: Notice that the rate-distortion region in Theorem [3] is achievable with a single 
round of interactions K = 1, which implies that multiple rounds do not improve the rate- 
distortion region in this case. 

Remark 11: It is worth mentioning that cooperation between encoders reduces the rate needed 
to communicate the source X 2 while increasing the optimization set of all admissible source 
descriptions. 

Proof: The direct part of the proof follows by choosing: 

^3-H2,i = C4-)-3,i = Ul—¥ 2 ,l = U 2 -H,l — U 3 -H,l = ^3->2 ,1 = 0, V/ 

Ui^ 23 =Ui^ 23y i = Xi , U\^ 23j i = U 2 ^i 3} i = U 2 ^ 3} i = 0 V l > 1 . 
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and C/" 2_>-13,1 = >13 and t/ 2 — >3,1 = f/ 2 _>. 3 are auxiliary random variables that according to 

Theorem |T| should satisfy: 

^2—>13 -e ~ ( Xl ,x 2 ) -e- X 3 , f/ 2 ^3, -^-(^13,X 1 ,X 2 ) -e- X 3 . (101) 


Notice, however that these Markov chains are equivalent to (95). From the rate equations in 
Theorem [I] and the above choices for the auxiliary random variables we obtain: 


R 1 ^ 23 >H{X 1 |X 2 ) , (102) 

R 2 ^ 13 >m ax {I(X 2 -U 2 ^ 13 \X 1 ),I(X 2] U 2 ^ 13 \X 1 X 3 )} (103) 

=I(X 2 ,U 2 ^ 13 |X 3 ) , (104) 

Ri^ 23 + R 2 ^i3>H(X 1 \X 3 ) + /(X 2 ; U 2 ^ 13 \XtXa) , (105) 

i? 2 ^ 3 >/(X 2 ;f/ 2 ^ 3 |(7 2 ^ 13 X 1 X 3 ) . (106) 


Noticing that R x = R\^ 2 z and R 2 = R 2 =, 13 + A 2 _>. 3 the rate-distortion region ( |34| ) reduces to 
the desired region in Theorem [3} where for simplicity we dropped the round index. We now 
proceed to the proof of the converse. 

If a pair of rates (i? 1; R 2 ) and distortions (_D 12 , /9 32 ) are admissible for the A'-stcps interactive 
cooperative distributed source coding setting described in Fig. [3j then for all e > 0 there exists 
7i 0 (e,K), such that Vn > n {) (e. K) there is a A'-stcps interactive source code (n. K. T. Q) with 
intermediate rates satisfying: 


-I>g Ill'll < Ri + e , *£{1,2} (107) 

1=1 

and with average per-letter distortions with respect to the source 2 and perfect reconstruction 
with respect to the source 1 at all nodes: 


E 


d(X?,X ») 


< D 32 + £ , 


Pr X^X” < £ , 


E 


d(X%,X. 


12 > 


<D l2 +£, 


Pr (X” 7 ^ X 31 ) < e , 


(108) 

(109) 

( 110 ) 
( 111 ) 
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where 


A 32 — #32 I <#l ’ <#2 j ^3 


x^n _ / /7-[l:^] x^n \ x^n _ / J y* 

^31 — #31 l <#1 ) i #2 ) ^3 ) ) ^21 — #21 I J \ i - A -2 

For each t G {1,... , n}, define random variables f/ 2 ^i 3 [t] and [/ 2 ^ 3 [t] as follows: 

*/ 2 -13[t] = ^ 2 [1: ^,Xi[i: t _i],X 1[<+1 : n] ,X 3 [i: t _l]) , 

^2->-3[t] — (f^2-)-13[t]5 ^3[t+l:n]) -^2[l:i-l]) • 


, X^ = g 12 [j\ l ' K \x^ 


rll*] 


( 112 ) 

(113) 


(114) 

(115) 


The fact that these choices of the auxiliary random variables satisfy the Markov chain ( |95| ) can be 
obtained from point 6 ) in Lemma 10 By the conditions ( | 111 [ ) and ( |109| ), and Fano’s inequality, 
we have 


H(X\' |X 3 ”i) < Pr (A'” # A 3 “) log 2 (||A 1 ”|| - 1) + H, (Pr(Af # AJ,)) = ne 
»(Ar|A” ) < Pr (A," + A” ) log 2 (|| Am - 1) + H 2 (pr(A,” / A” )) A « 

where e n (e) —> 0 provided that e —» 0 and n —)■ oo. 

1) Rate at node 1: For the first rate, we have 

n(i?i+e) >H(jl l:K] ^ 

> H 

(a) I ^j[l:K], X n| X nj 

= nH(X 1 \X 2 ) - H (x?\X2 j1 1:K]> ) 

> nH(X 1 \X 2 ) - H(X?\X”) 

>n[H{X 1 \X 2 )-e n ] , 


(116) 

(117) 


(118) 

(119) 

( 120 ) 
( 121 ) 

( 122 ) 

(123) 


where 


step (a) follows from the fact that by definition of the code the sequence j \ 1 ' ^ is a function 
of the both sources (X",X^), 


step ( b ) follows from the code assumption in ( |1 13[ ) that guarantees the existence of a 
reconstruction function X 2] = g 2 \ ( j\ 1 ' A , X!J 


step (c) follows from Fano’s inequality in (|117[). 
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2) Rate at node 2: For the second rate, we have 

n(R 2 + e) > H ( 


(jO t ( /tW'-K], vn vn vn 

— 1 I 2/2 

(b) 


(e) 


L iM 


> I (j] 1 :jq ;X?X?\X? 

= I (j} 1 :K] j} 1 :K] ;X?XZ\X{ 

= i ( V , [ 1 : K] j\ X '' K] \ x% | xr ) + / [j\ l:K] j^ Kl , x 2 n | x?x% 

n 

( =’E [^jrjn-.x 3l ^,x 3l , t _^ 

t= i 

+/ (jf ^ jf* 1 ; 

n 

Y l 1 (^l [1:A V 2 [1 ^lYl[l:i-l]X 1[i+1:n] X3 [1:f „ 1]; X3 [t] |X : 

t=l 

+ I (j} 1:K] jt K] X 1 [i:t-i] *2[t]|*i [t] * 3[t] 

n 

= 5] [/ (t/2^i 3[t] ;X 3[t ]|X 1[t] ) + / (C/ 2 _i3[ t] ;X 2[t] |X 1[t] X 3[t] ) 

t =1 L 

+/ (C/ 2 ^.3[ t j; X 2 [ ( ] |^ 1 [t]^3[t]C/ 2 ^.i3[t]) 
n 

(^2->13[t];^2[t]^3[t]|^l[t]) + I {U 2 ^3[t]', X 2[t] \X l[t] X 3[t] U 2 ^ 13[t] ) 

t=l 
n 

(^i3[t];X 2W |Xi W ) + / (c/ 2 ^ 3 [t]; 

t=i 
n 

= y [/ (^I3[«i; A' 2IQ] |A' 1BI ,0 = t) 

t=l L 

+/ (c/ 2 -73[qi ; x 2 [ Q ]\x 1[q] x 3[q] u 2 ^ 13[q] , Q = t ) 

w 

> n 


/ C/ 2 _>. 13 ; X 2 |X! + / f/ 2 ^ 3 ; Xal^Xa^is 


(124) 

(125) 

(126) 

(127) 

(128) 


(129) 


(130) 


(131) 

(132) 

(133) 


(134) 

(135) 


where 

• step (a) follows from the fact that 27) 1 ' A ' is a function of the sources (A"", X.]), 

• step ( b ) follows from the non-negativity of mutual information, 

• step (c) follows from the fact that J^' K ^ is a function of j \ 1 ' /x and the source A[\ 
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• step (d) follows from the chain rule for conditional mutual information, 

• step (e) follows from the memoryless property across time of the sources (X^X^Xg), 

• step (/) follows from the chain rule for conditional mutual information and the defini¬ 
tions ( |114| ) and ( |115[ ), 

• step (g) follows from the Markov chain C/ 2 —>-i 3 [t], X 2 [ t ])-e-X 3 ^, for all t e {1,..., n}, 

• step (h) follows from the use of a time sharing random variable 0 uniformly distributed 
over the set { 1 ,..., n}, 

• step (i) follows by letting new random variables C/ 2 _>. 13 = (U 2 ^i 3 [q\,Q) and U 2 ^3 = 

(U 2 ^3[Q],Q)- 

3) Sum-rate of nodes 1 and 2; For the sum-rate, we have 


n{R x + R 2 + 2e) > H {j\ X '' K] ^ + H ( J. (136) 
= H (jl 1:K] j\ V - K] ) + I jt K] ) (137) 

= / (j} 1:K] j 1 1:K] -, X?X%xf) + / (j} 1:K] -, jt K] ) (138) 

> / [j\ X:K] J^ K] \X\ l XfX!^ (139) 

= I (j} UK] J± 1:K] i X?\xf) + I [j\ X '- K] J^ K] ] X2\X?xf) (140) 

= h ( a ; i | x ”) - h (xfj\ x ' J<] j [x '- K] x^ 

+/ (j} 1:K] Jt K] -, X2\X?X?) (141) 

> H (X?\Xf) - H(XfXf) + I X2\X?xf) (142) 

> n [H (AilAs) -e n ] + I ; X2\X?xf) (143) 

n 

= Z 1 (^i [1:i " ] ^2 [1 ^ ] ^i[i: t -i]3f 1[ t +1:n] A3 [1:t _ 1] A 3[t+1:n] A 2[1:t _ 1] ; X 2[t] \X 1[t] A 3[t] ) 
t =1 

+ n[H( X 1 \X 3 )-e n ] (144) 

n 

= n [H (A,|X 3 ) ~e n ] + J2l (U 2 ^ 13[t] U 2 ^ 3[t] -X 2[t] \X 1[t] X 3[t] ) (145) 

t =1 


(9) 


= n [H(X,\X 3 )- e n + I (U 2 ^ Q f^ m -X 2 m \X m X m ,Q)} 


W 
= n 


H {X 1 \X 3 ) - e n +I (U 2 ^ 13 U 2 ^ 3 ; X 2 \X,X 3 


(146) 

(147) 


where 
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step (a) follows from the fact that j\ Lh and j \ 1 ' A ' are functions of the sources (X™, X 2 , X.”), 
to emphasize 

step (6) follows non-negativity of mutual information, 


step (c) follows from the code assumption in ( |113| ) that guarantees the existence of recon¬ 
struction function = g 3l J^' K \ X% j, 

step (d) follows from Fano’s inequality in ( | 11 1 | ), 

step (e) follows from the chain rule of conditional mutual information and the memoryless 

property across time of the source (X", Xg, Xg), 

step (/) from follows from the definitions ( |1 14| ) and ( |115| ), 

step (g) follows from the use of a time sharing random variable 0 uniformly distributed 
over the set {1,..., n}, 

step (h) follows by letting new random variables f/ 2 -s.i 3 = (£4->i3[Q], Q) and C / 2 ->3 = 
(U 2 ^3[Q],Q)- 


4) Distortion at node 1: Node 1 reconstructs a lossy X" 2 = g 12 ( J 2 


r[l:K] 




It is clear that 


we write without loss of generality Xf 2 = g v > yj\ XK \ J 2 ' K \ X"j. For each t G 
define a function Xi 2 [t] as beging the /-1h coordinate of this estimate: 

X\ 2 [t\ {U 2 ^ m ,x 1[t] ) 4 g m (j\ v - K \j^- K \x^ . (148) 

The component-wise mean distortion thus verifies 

D 12 + e > E [d (X 2 , g 12 , X ?)) 




t =1 


- ^ E d (X 2 [Q] , X [2 [Q] (U 2 ^ 13 [ Q \, Xi [Q ]) 


t= 1 


Q — t 


= E 
= E 


d (X 2[Q] ,X 12[Q] (f/ 2 ^l3[Q],A"i[Q]) 
d (x 2 , X 12 (U 2 ^X 1 


(149) 

(150) 

(151) 

(152) 

(153) 


where we defined function X i2 by 

X 12 (*7 2 _> 13 , X,) = X 12 (Q, U 2 ^ 13[Q] ,X 1[Q] ) 4 x im X 1[Q] ) . (154) 
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5) Distortion at node 3: Node 3 reconstructs a lossy description X 32 = g 32 , j \''^, X 3 

For each t 6 {1,..., n}, define a function X 32 [ t ] as beging the f-th coordinate of this estimate: 


x 3 2[t] (U2^m,u^ m ,x m )±g m (j1 1:IC] ,j1 1:K] ,xz 

The component-wise mean distortion thus verifies 

D 32 + e > E \d (x 2 ,g 32 (j} 1 :K] ,ji 1:K] , X£) 


n 

1 

n 


^ E d f X 2 [ t ], X 32 [ t ] (U 2 ^i 3 [ t \i U 2 ^ 3 [t], X 3 [ t ]) 


t= l 


d ^A" 2 [Q], A 32 [Q] (U 2 -+13[Q],U 2 -+3[Q],X 3 [Q]) 

—1 

d (X 2 [Q] ,X 32[Q] (U 2 ^ 13 [Q] ,U 2 ^ [Q] ,X 3[Q] ) 
d(x 2 , X 32 (U 2 ^ 3 ,U 2 ^ 3 ,X 3 


Q — t 


= E 
= E 

where we defined function A" 32 by 

x 32 (u 2 _ >i 3 , U 2 _, 3 , A" 3 j = X 32 (Q, U 2 ^i 3 [q], U 2 -+3[Q], X 3 [Q]) 

— *32[Q] (U 2 _H3[Q], U 2 _>3[Q], X 3 [Q]) . 

This concludes the proof of the converse and thus that of the theorem. 


(155) 

(156) 

(157) 

(158) 

(159) 

(160) 


(161) 


C. Two encoders and three decoders subject to lossless/lossy reconstruction constraints, reversal 
delivery and side information 

Consider now the problem described in Fig. [4] where encoder 1 wishes to communicate the 
lossless the source X™ to node 2 and a lossy description to node 3. Encoder 2 wishes to send 
a lossy description of its source Xj to node 1 and a lossless one to node 3. The corresponding 
distortion at node 1 and 3 are D\ 2 and D 31 , respectively. In addition to this, the encoders 
accomplish the communication using K communication rounds. This problem is very similar to 
the problem described in Fig. [3j with the difference that the decoding at node 3 is inverted. 

Theorem 4: The rate-distortion region of the setting described in Fig. [4] is given by the union 
over all joint probability measures Px 1 x 2 x 3 u 2 ^i 3 satisfying the Markov chain 

U 2—>. 1 3 -©- (X\, X 2 ) -e- X 3 (162) 
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Figure 4: Two encoders and three decoders subject to lossless/lossy reconstruction constraints, 
reversal delivery and side information. 


and such that there exists reconstruction mappings: 


^31 {X 2 , Xs, £/ 2 -> 13)=^31 with 

E 

d(X U X 31 ) 

< Ail , 

(163) 

.9 12 (Ai, U 2 ^ 13 )=X 12 with 

E 

d(X 2 ,X l2 ) 

< D 12 , 

(164) 

of the set of all tuples satisfying: 





Rx>H{X 1 \X 2 ) , 




(165) 

R2>I(U 2 -+13]X 2 \X 1 ) + H(X 2 \U 2 ^ 13 X 1 X 3 ) , 

(166) 

R\ + R 2 >H(X x X 2 \X;) . 




(167) 


The auxiliary random variable has cardinality bounds: ||ZY 2 —s>i 3 1| < ||Tj || \\X 2 \\ +3. 

Remark 12: Notice that the rate-distortion region in Theorem [4] is achievable with a single 
round of interactions K = 1, which implies that multiple rounds do not improve the rate- 
distortion region in this case. 

Remark 13: Notice that, although node 3 requires only the lossy recovery of Xi, it can in 
fact recover X\ perfectly. That is, as node 3 requires the perfect recovery of X 2 , it has the same 
information that node 2 who recover Xi perfectly. This explains the sum-rate term. We also see, 
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that the cooperation helps in the Wyner-Ziv problem that exists between node 2 and 1, with an 


increasing of the optimization region thanks to the Markov chain (162). 
Proof: The direct part of the proof follows by choosing: 


U3-+12,1=Ui^ 3 j = Ui->2,1 = U 2 -+l,l = ^3-s-l,/ = ^3^-2,Z = 0, V/ 

>23 = ^1—>23,1 = Xi , C/ 2 -> 3 = C^2—>3,1 = X 2 U\^ 2 3,l = U 2 -+V&,1 = (^2->3,Z = 0 V l > 1 

and with {7 2 ->i3,i = U 2 ^i 3 auxiliary random variable that according to Theorem [I] should satisfy: 

^2—>13 -e_ (X 1; X 2 )^-X 3 . (168) 

From the rate equations in Theorem |T[ and the above choices for the auxiliary random variables 
we obtain: 


R l ^ 2 3>H(X 1 \X 2 ) , (169) 

-R 2 -> 13 >max {/(X 2 ; U 2 ^ 1 3 \X 1 ), J(X 2 ; U^X^)} (170) 

=I(X 2] U 2 ^i3\X 1 ) , (171) 

Ri ^23 + R 2 ^i3>H(X 1 \X 3 ) + I(X 2 ; U 2 ^i3\X 1 X 3 ) , (172) 

R 2 ^ 3 >H(X 2 \U 2 ^ 13 X 1 X 3 ) . (173) 


Noticing that R,\ = i?i^. 23 and R 2 = R 2 ^i 3 + i ? 2 ^.3 the rate-distortion region (J34J) reduces to 
the desired region in Theorem [4] where for simplicity we dropped the round index. We now 
proceed to the proof of the converse. 

If a pair of rates (Ri, R 2 ) and distortions (D\ 2 , D 3 1 ) are admissible for the K -steps interactive 
cooperative distributed source coding setting described in Fig. [4} then for all e > 0 there exists 
n 0 (e,K), such that Vn > n 0 (e,K) there exists a K -steps interactive source code (n, K.T.Q) 
with intermediate rates satisfying: 

, K 


n 


log ||27'/|| < R{ + e , i e {1,2} 


(174) 


z=i 
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and with reconstruction constraints: 


where 



E 

d(X r,AJi)_ 

< D'i i + £ , 


(175) 


Pr 

(x r ^ Xn) 

< £ , 


(176) 


E 

d(A 2 ”,A”)‘ 

< D 12 + £ , 


(177) 


Pr 

(x’i # Xi) 

< £ , 


(178) 

III 

CO 

to 


, M 2 = .912 


(179) 

w S 

III 

00 

(j1 1:K] ,j1 1:K] ,xz) 

, ^2, = .921 


(180) 


For each t G {1,..., n}, define random variables f/ 2 —>] 3 [t] as follows: 

TT — ( 'tW'K\ V Y Y 

t-^2—>13[t] — l J\ iJ2 >^-l[l:t-l],^-l[t+l:n],^-3[l:t-l] 


(181) 


Using point 6) in Lemma 10 we can see that this choice satisfies ( 162[). By the conditions ( |176| ) 


and (jTT8J), and Fano’s inequality, we have 

< Pr (*T + *2i) lo&dWII - 1) + H 2 (Pr(A'" / A” )) 4 n£ „ , (182) 

//!AJ|A;y < Pr (AJ / A 3 “ 2 ) logjdlAfU - 1) + H 2 (pr(A 2 ” + A 3 ” 2 )) 4 , (183) 

where e n (e) —> 0 provided that e —>• 0 and n —* oo. 

1) Rate at node 1: For the first rate, from cut-set arguments similar to the ones used in 
Theorem [3] and Fano inequality, we can easily obtain: 

n(Ri +e)>n [H(X 1 \X 2 ) - e n \ (184) 
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2) Rate at node 2: For the second rate, we have 


n(R 2 + e)>H[J\ 


(+) t ( /tW'-K], vn vn vn 

— 1 \J2 


>i(Jt hx 2 x^ |A7 


( J / (j} 1:K] j1 1:K] -x2x%\x? 


= I (j} 1:K] jt K] -,x%\x?) +i (j} 1:K] j} 1:K] -,x 2 \x?xz 


(185) 

(186) 

(187) 

(188) 
(189) 


n 

E [ 7 (jf'+f* 1 ; Wml^r, Wn.^1] 


+/ ( J\ 1:K] Jt KI ; Xwix?x?x m - tl 


(190) 


= J2 i 1 (jl I1 ^^ 1: ^lN-l]^l[«-l:»l^S[l ; l-li;^3ll]|- y 1| < ] ) 

t= 1 

+ I (j 1 [1: ^ ] ji 1: ^ ] X 1[1:t _ 1] X 1[t+1:n] X3 [1:t _ 1] X3 [t+1;n] X 2[1:4 _ 1] ;X 2W |X lW X3 W ) ](191) 

n 

— j^7 ((7 2 ^i3[t];96 3 [t]|96 1 [t]) + 7 ((7 2 ^.i3[ t ]96 3 [ t+1 . n ]96 2 [ 1: (_ 1 ];96 2 [ t ]|X 1 [ t ]96 3 [ t ]) 

t =i L 
n 

(U 2 -+ m ; 96 2 [ t ]X 3 [ t ] I A"i[i]) 


+7 (96 3 [ t+ i :n ] A" 2 [ 1:t _!]; A" 2 [ t ] |96i[ t ]96 3 [ t ] C/ 2 

n 

= 5] [i (U 2 -> m -,X 2[t] \X 1[t] ) + H (X 2[t] \X 1[t] X 3[t] U 2 ^ m ) 

t =1 

—77 (96 2 [ t ]|96i[i]96 3 [ t:n ]C/ 2 ^.i 3 [t]96 2 [i :t _i]) 

(/l) 12 

> ^ [7 ((7 2 ^i3[ t j; X 2 [i]|A"i[ t] ) +77 (^ 2 [t]|^i[t]^ 3 [t]£7 2 ->.i3[t]) — e n 

t =1 L 
n 

= E[ 7 +-i3|«i;A' 2IQ] |A' 1M ,Q = «) 

t=l 

+77 ( k X2[q]\Xi[q]X 3 [q\U 2 ^i3[q], Q = 7) — e n 

> nf7 + 77 fx 2 |X 1 X 3 £7 2 _ 13 > ) - ej , 


(192) 


(193) 


(194) 


(195) 


(196) 


where 
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step (a) follows from the fact that j \ 1 ' ^ is a function of the sources (X™, AT)'), 
step ( b ) follows from the non-negativity of mutual information, 


step (c) follows from the fact that j\ 2 ' h is a function of n and the source A", 
step (d) follows from the chain rule for conditional mutual information, 
step (e) follows from the memory less property across time of the sources (X[\ X 2 , X”), 


[i -K) 


• step (/) follows from the definition ( |1 81 [ ) 

• step ( g ) follows from the Markov chain U 2 ^iz\t} -e-(Xi[ t ], X 2 [t])-e-X 3 [ t ], for all t G n} 

and the usual decomposition of mutual information, 

• step (Id) follows from the fact that Pr (a f 2 [t] 7 ^ X 32 [t] j < e Vf G {1, , X 32 [f] = 

^ 32 [t](^i 3 [t],^ 3 [t:n]) and Fano inequality. 

• step (i) follows from the use of a time sharing random variable 0 uniformly distributed 
over the set {1,..., n}, 

• step (j) follows by defining new random variables t / 2 ->13 = (U 2 ^i 3 \q\,Q). 

3) Sum-rate of nodes 1 and 2: From cut-set arguments and Fano inequality, we can easily 
obtain: 

n(i2r + R 2 + 2e) > n [H(X\\X 2 X 3 ) - e n } 


4) Distortion at node 1: Node 1 reconstructs a lossy Xf 2 = g 12 X" 

t G {1,..., n}, define a function A 12 m as the f-th coordinate of this estimate: 

dd]2[t] {U 2 ^ m] ,X 1[t] ) ^ g m . 

The component-wise mean distortion thus verifies 

D 12 + e > E \d (X2, 912 (J1 1:K] , X?) 


n 

1 

n 


= E 
= E 


^^E d\X 2 [ t ], Xi 2 [ t ] (U 2 ^ 13 [ t ], 

t =1 
n 

X^E d (x 2[Q] ,X 12[Q] (U 2 ^i 3 [q\, X 1 [ Q ]) 

t =1 

d ( X 2 [Q], X 12 [Q] (U 2 ^ 13 [Q], Xl[Q]) 


Q — t 


d[X 2 ,X l2 [U 2 ^ 13 ,X 1 


where we defined function Xi 2 by 

A12 (U 2 ^ l3 , Xij = X12 (Q,U 2 ^i 3 [Q],Xi[Q]) = A"l2[Q] (U 2 ^13[Q\, X±[Q]) . 


(197) 
For each 

(198) 

(199) 

( 200 ) 

( 201 ) 

( 202 ) 

(203) 

(204) 
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5) Distortion at node 3: Node 3 reconstructs a lossy description X”, = g 31 (j) 1 '^, Xg'j . 
For each t G {1,..., n}, define a function X 31 [ t ] as: 

^3i[t] (u 2 ^ m , X m , x m , X 3 [ t+1:n] ) = .9.3 1[t] (^ [1:K] , jf* 1 , * 3 n ) n}. (205) 

This can be done because X 32 [ t ] is also a function of J 2 ' K \ X %\. The component-wise 


mean distortion thus verifies 


D 32 + e > E [d (AT ,gsi(ji 1:K \ji 1:K] ,X^y 

1 U 

— ~ ^^E ^ (-^l[t]j -^31 [t] (^2->13[t], Xz2[t], -^3[t]j -^3[t+l:n] ) 
U t= 1 

1 n 

“TEX Aw ) -^31[t] [U 2 —>-13[t ]i X 2 [i] , -^3 [t] J -^3[t+l:n 1 ))] 


> d (a iW , A 31[t] (f7 2 -^i 3 [t] ,^2[t],^3[i]))j 

1 t =1 

1 n 

= - X/ ^ (^1[Q]> ^31 [Q] (^2-*13[Q], ^2[Q], ^3[Q])) <5 = t 

L t= 1 

= E d (a7[ Q] , A 32 [q] (t/ 2 —>i3[q], A" 2 [q],X 3[ q])) 

= E [d (A7,X 32 (t/ 2 ^ 13 , A 2 , A 3 ))] , 


(206) 


(207) 


(208) 


(209) 


( 210 ) 


( 211 ) 

( 212 ) 


where 


step (a) follows from the fact that X 31[i ] ^U 2 ^ 13[t] , X 32[t] , X 3[t] , X 3[t+1:n] ) can be trivially 
expressed as a function of (U 2 ^ 13[t] , X 2[t] , X 3[t] , X 3[t+1:n] ) as follows: 

- , „ x I ^31 [t] {U 2 ^i 3 [t]X 2[t] X 3[t] X 3[t+1:n] ) if x m = X 32 [t] 

A 31[l] X2^13[t]^2[t]^3[t]^3[t+l-.n]) = \ - / ~ \ 

( -^31 [t] (^2^13[t]^32[t]-^3[t]-^3[t+l:ra]J if A 2 [ t ] T X 3 2[t] 

step (6) follows from the fact that A 1[t] -e- (£/ 2 ->-i 3 [t]^ 2 [t]^ 3 [t]) A7 [t+I:r) |, which implies 

that for all t G (1,... ,n} exists X 3 i [t] (U 2 ^ 13[t] ,X 2[t] , X 3[t] ) such that 

E d (Ai[jj, A 3 i[ t ] (f/ 2 _ 5 >i3[t]X 2 [t]A 3 [ i ]) j < E [d (Ai[ t j, A 3 i[ t ] (f/ 2 _ 5> i 3 [ t ]A 2 [t]A 3 [ t ]A 3 [ t+ i ;n ]))] 

We also defined function X 32 by 


(213) 


A 32 (U 2 ^ 13 ,X 2 ,X 3 ) — A 32 (Q ,U 2 ^i 3 [q], X 2 [q\, X 3 [q]) 

- X 32 [Q] {u 2-». 13 [Q], A 2 [Q], A 3 [Q]) . 


This concludes the proof of the converse and thus that of the theorem. 
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D. Two encoders and three decoders subject to lossy reconstruction constraints with degraded 
side information 


Consider now the problem described in Fig. [5] where encoder 1 has access to X x and A" 3 and 
wishes to communicate a lossy description of X\ to nodes 2 and 3 with distortion constraints 
Z} 2 i and £> 3 i, while encoder 2 wishes to send a lossy description of its source Xlj to nodes 1 
and 3 with distortion constraints £> 12 and D 32 . In addition to this, the encoders overhead the 
communication using K communication rounds. This problem can be seen as a generalization of 


the settings previously investigated in fl9| . This setup is motivated by the following application. 
Consider that node 1 transmits a probing signal X 3 which is used to explore a spatial region (i.e. 
a radar transmitter). After transmission of this probing signal, node 1 measures the response (Ay) 
at its location. Similarly, in a different location node 2 measures the response A" 2 . Responses X\ 
and A" 2 have to be sent to node 3 (e.g. the fusion center) which has knowledge of the probing 
signal AT and wants to reconstruct a lossy estimate of them. Nodes 1 and 2 cooperate through 
multiple rounds to accomplish this task. 





Figure 5: Two encoders and three decoders subject to lossy reconstruction constraints with 
degraded side information. 


Theorem 5: The rate-distortion region of the setting described in Fig. [5] where A!-e-A 3 -e-A 2 
form a Markov chain is given by the union over all joint probability measures Px 1 x 2 x 3 w [1 h ._ l] r/ I ^ 3 K 
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satisfying the following Markov chains: 


Ui^.23,i (-X"i, X 3 , W[i,/]) -e- X 2 , 

U 2 -+w -e- (X 2 ,W[2a) (^1,^3) , 

-e- (*1, X 3 , W [2>J q) -e- X 2 , 

for all l — [1 : K], and such that there exist reconstruction mappings: 

gi2{Xi,Xz,U\-^z t K,yV[i,K+\}) = X12 with E d(X 2, *12) < -D 12 , 

921 (X2, yV\i,K+i\) — X 2 i with E d(Xi,X 21 ) < D 2 \ , 

931 (X3, Wfi^+i], U\^k) — *31 with E d{X i,* 3 i) <-D31 , 

932 {Xs, Wji^+i], U\^,k) — *32 with E d( A 2 ,X 32 ) < -D 3 2, 

with Wfi^j = {Ei_>. 23i z, t/ 2 ^i 3: /}^'|for all l — [1 : K] 9 of the set of all tuples satisfying: 


(214) 

(215) 

(216) 


Ri > I^k+i^X.X^) + I(U 1 ^ K -X l \W [1 , K+1] X 3 ) , (217) 

R2>I(W[i,k+ i]]X 2 \X 3 ) . (218) 

The auxiliary random variables have cardinality bounds: 

1 -1 

||Wi_ 23| i|| < 11*11111*311 n 11^23, illllWa—,13, ill + 1, le[ 1 : A'] (219) 

i =1 
/-I 

||A/ 2 —^13,Z II < ||*2||||Wi_>.23,i|| 11^1—^23,* II 11^2^-13,11| + 1; ( G [1 : A] (220) 

i= 1 

K 

||Wi_^,,|| < 11*31111*31^11^1111^13,11 + 3. (221) 

i =1 


Remark 14: Notice that multiple rounds are needed to achieve the rate-distortion region in 
Theorem [5] It is worth to mention that first encoders 1 and 2 cooperate over the K rounds while 
on the last round only node 1 send a private description to node 3. Because of the Markov chain 
assumed for the sources we observe the following: 

. Only node 1 send a private description to node 3. This is due to the fact that node 3 has 
better side information than 2. 


9 Notice that = 0 for all l because R 3 = 0. 
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. For the transmissions from node 2, both node 1 and 3 can be thought as an unique node 
and there is not reason for node 2 to send a private description to node 1 or node 3. 

. Notice that the there is not sum-rate. Node 3 recovers the descriptions generated at nodes 
1 and 2 without resorting to joint-decoding. That is, node 3 can recover the descriptions 
generated at nodes 1 and 2 separately and independently. 

Proof: The direct part of the proof follows by choosing: 


— ^3->12,Z — ^3->l,Z — U 3 ^2,l ~ ^2->3,Z ~ V/ 

*7i—> 3 ,z=0 l < K . 

and U i—> 23 ,z and C/ 2 —s-i 3 ,z and U\^,k are auxiliary random variables that according to Theorem 
[T] should satisfy the Markov chains ( |214[ )-( [2T6| ). Cumbersome but straightforward calculations 
allows to obtain the desired results. We now proceed to the proof of the converse. 

If a pair of rates (f?i, Rf) and distortions ( D\ 2 . D 2 1 , D :il , Dmi) are admissible for the A'-stcps 
interactive cooperative distributed source coding setting described in Fig. [5] then for all e > 0 
there exists n 0 (e,K), such that V/z > n 0 (e,K) there exists a A'-stcps interactive source code 

(n, A', J 7 , (7) with intermediate rates satisfying: 

„ k 


yi l°g \\Ji\\ < Ri + £ , i e {l,2} 


z=i 


and with average per-letter distortions 


( 222 ) 


E 

d(X?,X ») 

< £*21 + £ , 

(223) 

E 


< -D31 + ^ , 

(224) 

E 

d(x 2 -,xr 2 )" 

< £*12 + £ , 

(225) 

E 

4A7,A7 2 )‘ 

< Z7 32 + e , 

(226) 


where 


yn — I /r[l -K\ rr[l-K\ 
-^32 — $32 I iJ 2 : 


\rn 


yn _ I rr\y-.K\ :K\ yn yr 

-'■12 — $12 l J\ i J 2 , Ai,A 3 


yn _ I rr[l'.K\ rr\l-.K\ yn 

^31 — ^731 l , >$2 ; ^-3 


^ I r r\y~- K \ q\y- K \ yn 

■A-21 — 9 21 I J 1 J C 2 ; ^-2 


(227) 

(228) 
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For each t G {1,..., n}, define random variables t/ 2 —> 3 ,[t]) and the sequences of random 

variables (C/i_> 2 3,fc,[t], U 2 ^i 3 ,k,[t])k=[i:K] as follows: 


tfl->23,l, [t] — (Jl,-X 3 [i :t _ 1 ],X 2 [t+l: n ]) , 

(229) 

^2—>13,1,[t] — Jl i 

(230) 

tfi->23,fc,[t] — Ji , V A; = [2 : K\ , 

(231) 

tf2-n3.fc.it] = , V k = [2 : A'] , 

(232) 

tfl->3,A',[t] — ^3[t+l:n] > 

(233) 


From Corollary [4] in the Appendices we see that these choices satisfy equations (214), (215) and 

d2T6b . 


1) Rate at node 1: For the first rate, we have 


n(Ri + s) > 

(a) 

(b) 
> 

(c) 


(J 


69 

> 


+ 


H (j]' K] ) 

(234) 

i (j}' K ■. x';x:;x’:) 

(235) 

i(jl hK] -,x?xz |x 2 ") 

(236) 

i (jl 1:K] jt K> -,x rxjix") 

(237) 

I\j'i i:K] Ji hK] -X; l-YJ.Yj”) + / (jf* 1 jf ,f| ; JfJIXj) 

(238) 

n 

Y 1 (jl 1:K] jt K] X2[l: t -l] ^[t+UnJ^I^t-iJ^im.nJ^Ii.t-!]; ^[tll^M^It]) 

t=l 

71 

5] / J 2 [1:K1 X 3[1:t _ 1] X 2[1:t _ 1] X 2[t+1:n] ; X 3[t] |X 2W ) 

t=l 

(239) 

n 

^/(j 1 [1 ^ 1 J 2 [1:K1 X 2[t+1:n] X 3[1: ,_ 1] X 3[t+1:n] X 1[1:t _ 1] ;X lM |X 2W X 3M ) 

t=l 


n 

Y 1 ( l 7i [1: ^ ] j:J 1:K1 X3[i: i _,]X 2 [, + 1 : n] ;X 3W |X 2 [ t] ) 

t=l 

(240) 
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t=l 
n 

+ Y, i 1 (^3[ t+ l:n]^ 1[1 : t - 1 ];^ lW |Jl [1:K] J2 [1: ^^2M^3[l:t] 

i=l 

+/ (^ 1 [1:/f] ^ 2 [1: - K] X 3[1:t _ 1] X 2[t+1:n] ; X 3[t] |X 2W 

n 

= (^l [1;A Vi 1:A lY 2 [ t+ l : n]X3 [1:t _ 1]; X lW X3 W |X 2W 

t=l 
n 

+ ^ J (^3[ t+1: n ] ^ 1 [ 1:t - 1] ;^ 1 [ t] |Jl [1:ifl J 2 [1:K1 ^ 2 M^3[ 1: t ] 

£=1 
n 

= J2 I { J i 1:K]j ^' K]X ^+ l : n]^3 [ l:t-l ] ;^ lW X 3W |X 2W 
t=l 
n 

+ ^ / (x 3[t+1:n] x 1[1:t _ 1]; ^ 1[t] J^ 2[t] |^ 1 [1: ^^ 1: ^J^ 2[t + 1: „ ] ^r 3[1 : t] 

t=i 

t=l 
n 

+ (x 3[t+1:n] ;X 1[t] | l 7 1 [1:A1 l 7 2 [1:A1 X 2[ i + i : n]X 3[ i : t] 


t=i 


- 5Z 7 (^23,[l:K],[t] C/2^13,[l:^],[ t] ; -^l[t]^3[t]|^2[t]) 
t= 1 
n 

+ I (Ul^3 t K,[t]]Xi[ t ]\U 1 ^23,[l:K],[t]U2^13,[l:K],[t]X3[ t ]) 


t= 1 
n 


V\ I (^l^ 2 3,[l:K],[Q]C4->-13,[l:/f],[Q];^l[Q]^ 3 [Q]|^2[Ql)..Q ~ f) 

t=l 

n 

I {U 1 ^KM'’Xl[Q]\U 1 ^23Al:K]M U ^mmM X m’Q = 0 


t=l 


(241) 


(242) 


(243) 


(244) 


(245) 


(246) 


(*) 

= n 


= n 


I [Ul-+23, [1:K]U2->13, [1:K}\ ^1^3^ 2 J +-^ [Ul-+3,K‘, Xi \ £/l->23,[l:.K] ^2-5>13,[l:A']^3 

/ (w [1 ,k +1] ;^3|^ 2 )+/ 


(247) 


where 

• step (a) follows from the fact that J7) L ^ is a function of the sources (A"", A")', X7), 

• step (6) follows from the non-negativity of mutual information, 

• step (c) follows from the fact that j\ 1 ' A is a function of j \ 1 ' /n and the source A")', 
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• step (d) follows from the chain rule for conditional mutual information and the memoryless 
property across time of the sources (Af, A^Ag), 

• step (e) follows from the non-negativity of mutual information, 

. step (/) follows from the Markov chain X 2[t] -e-{jl 1 ' K] ji 1:K] X 2 [t + i: n ]^ 3 [i : i ] )-e-(^ 3 [t+i : n ] ^i[i : t-i ] ) 
(Corollary [4] in the appendices), for all t — [1 : n] which follows from X\ -e- A 3 -e- X 2 ., 

• step (g) follows from the non-negativity of mutual information, 


. step (h) follows from defintions (233). 

• step ('/’) follows from the standard time-sharing arguments and the definition of new random 
variables, (i.e. Ui^\i-.k] = (E/i->23,[i:X],[Q], Q) and A, = (X 1[q] ,Q)). 
and the last step follows from the definition of the past shared common descriptions Wn n VZ. It 
is also immediate to show that £ 4 ->i 3 , i) satisfies the Markov chains in ( |214[ )-( j2l6| for 

all l e [1 : K]. 


2) Rate at node 2: For the second rate, by following the same steps as before we have 


n(R 2 + e) > H(ji 1:K] 


(a) 

(b) 

> 

(J 

(d) 


= /[J 2 [1:K| ;XrX 2 "X 3 ” 
[1:X] . 


> / ( j ™ a ? \x?x% 

'=' / ( j} 1:K1 j} 1:IC1 ;X2\x?x£ 
= I ( j} 1:K] j£ :K] X?- A 2 n |A 3 n 


> I (j} 1:K] j1 1:K] -,x?\xi 


(/) 


(ff) 


n 

^/(j 1 [1:K1 ^ 2 [1:X1 A 3 [i :t _i ] A3 [t+ i: n ]A 2[t+1:n] ;A 2[t] |A 3W ) 

t =i 




t=i 

n 


> J]/ (ji [1:K1 ^ 2 [1:K1 Jf3 [1: ^ 1] ^ 2[t + 1: n ] ; *2[t] 


E 1 (^l-^23,[l:X],[t]^2->.13 ! [l:K] ! [f];^2[t]|^3[t]) 


t=l 


(0 


= n/ Wp,*r +1] ;A 2 |A 3 


(248) 

(249) 

(250) 

(251) 

(252) 

(253) 

(254) 

(255) 

(256) 

(257) 


where 


step (a) follows from the fact that j \ 1 '^' is a function of the sources (A”, A 2 , A 3 ), 
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step ( b ) follows from the non-negativity of mutual information, 

step (c) follows from the fact that j\ { ' h is a function of J 2 ' K ^ and the source (Xj\ X' s '), 
step (d) follows from the Markov chain X\ -e- X 3 -e- X 2 . 
step (e) follows from the non-negativity of mutual information, 

step (/) follows from the chain rule for conditional mutual information and the memory less 

property across time of the sources (Xf, X!}. X"), 

step (g) follows from the non-negativity of mutual information, 


• step ( h ) follows from definitions (233). 

• step ( i ) follows the definition for W\u] V/ and from standard time-sharing arguments similar 
to the ones for rate at node 1. 

3) Distortion at nodes 1 and 2: Node 1 reconstructs an estimate X\ 2 = g 12 J 2 ' k . X[\ Xd 

while node 2 reconstructs X 2l = g 2 \ (j\ kK - J 2 ' K \ X ' 2 ^. For each t 6 {l,...,n}, define 
functions X\ 2 [t] and X 2 1 \t] as being the f-th coordinate of the corresponding estimates of X'[ 2 
and X ' 2l , respectively: 

x m X 1[t] , X 3[th X 1[1:t . lh X 1[t+1:n] ) 4 g m (j} 1:K \j£ :K \x?y?. 58) 

X 2 1 [tj (W [ltK+lUt] ,X 2[t] )±g 21[t] (jl V - K \j^ K] ,X^(259) 

The component-wise mean distortions thus verify 


D\2 + 


e>E[d (x 2 ,g 12 (j} 1:K] ,j} 1:K] ,X D) 


(a) 1 


- ^2 E ^ (x2[t] , Xi2[t] !],[*], Ai[ t ], X 3 [ t ], 2fi[ t+ i:„]) 


i=1 


(fc) 1 
> - 
n 


,K,[t], Xi[i\, X 3 [ t ]) 


t =1 


- ^ E d (x 2[Q ], X* 2[Q \ (W[i iA - + i ] . [ Q ] , Ux^k^q], A"i[Q], A 3 [ q]) 


t =i 


Q = t 


= E 


( = } E 


d {X 2[Qh X* 12[Q] (W[i jA '+ 1],[Q], ^1^3,A-,[Q]^i[g],2f 3 [Q]) 
d (x 2 ,Xi 2 (w [1 , k+ i ] ,U 1 ^ k ,X 1 ,X 3 


(260) 

(261) 

(262) 

(263) 

(264) 

(265) 


where 


step (a) follows from (259), 
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. step (6) follows from Markov chain X 2[t] -e- (X 1[t] , X 3[t] , J^' K \ ^ 3 [ 1: t-i], ^ 3[ i+i : n], 

X 2[t+1:n] ) -e- (X 1[1:t _ 1]: X 1[t+1:n] ) V t — [1 : n] (which can be obtained from Corollary [ 4 ] in 
the appendices) and Lemma [9j 
• step (c) follows from the following relations: 

A 12 (yV[i,K+i],Ui^ 3j K,Xi,X 3 ^j = X V2 (Q, W[i iJf+ i],[Q], £4^,[Q], Xi [Q] , x 3[g] ) 

- X 12[Q] (Wji.A’+il.IQ], Ux^K^Q], X 1[Q] , X 3[Q] ) . 


By following the very same steps, we can also show that: 


(266) 

(267) 


D 2l +e > E \d (X u g 2 ,(j\ x '- K \J^ K \X") 

= E[d (X U X 21 (w [ltK+1] ,X 2 

and where we used the Markov chain X 2 [i- t -x] -e- ( X 2 [ t ], Ji' K \ J 2 ' K \ ^ 3 [i:t-i], X 2 [ t+ i:n\) -e-Xi w 
V t — [1 : n] (which can be obtained from Corollary [4] in the appendices) and Lemma [9] and 
where we define the function X 21 : 

X 2 i x 2 ) =x 21 ( Q,W {1 ,K+1],[Q}’ X 2[Q ]) A X 2 1[Q] i^{\.K+l].\Q\, U 2 ^ v3[Q]l X 2[Q] ^ . 

4) Distortions at node 3: Node 3 compute lossy reconstructions X 31 = g 31 ( j\ 1 ' K . j\ 1 ' K . X 3 


and X-! a = g 32 J^' K \ X^y For each t e {1,... , n}, define functions X 3l ^ and X 32 [t\ 

as being the /-1h coordinate of the corresponding estimates of X 31 and X 32 , respectively: 

t 3 i[ t ] {W[i, K+ iUt], U^KM’Xm) = 9m (jl 1:K] ,Jl 1:K Ul) , (268) 

x m {W[1, K+ I],[t], U^K^Xw) 4 g. m (jl 1:K \jl VK \xz) . (269) 

The component-wise mean distortions thus verify 

D 31 + e>E[d (x u g :n (j} ] '- K] ,jt- K] ,XZ)) 


= E 
= E 


X] E d (Xi[ t] , X 31 [ t ] (W[1^+1], M , Ui^ 3 ,K,[t], X 3[i\) 

t= 1 
n 

E E d (x 1 [Q] ,X 31[Q] (W[i jjR r + i],[Q], C/i-^rqQ],-A 3 [qj) 

:=i 

d ( Al[Q], AT 3 1[Q] (W^k+IUQ}’ Ui-+3 t K,[Q], X 3 [Q]) 


Q — t 


d ( X }, X 31 ( W[i ; ^ + i], Ui-> 3j k, X 3 


(270) 

(271) 

(272) 

(273) 

(274) 
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where the last step follows by defining the function X 3l by 

X31 = -X 31 {Q,W[l,K+l],[Q],Ui^ 3 t K,[Q],X 3 [Q]) 

~ X 3 l[Q] (W[1,K+1],[Q], Ui_y 3i K,[Q], X 3 [Q]) 

By following the very same steps, we can also show that: 


D 32 + e > E 
= E 


<*(x 2 , 9 32 (J' 1 |,: ' C 1 ,J 2 | 1 : ' C 1 ,W”)) 
d (Xj. X 32 (VV| 1 ,* +1| , 0, ,:u f ,X 3 )) 


and where we define the function X 32 by 


(275) 

(276) 


X32 Ui^. 3jK , X 3 ^j = X 32 (Q,W[l,K+l],[Q],Ui^ 3 'K,[Q],X 3 [Q]) 

- X32[Q] (W[l,K+l],[Q],Ui^ 3 jKj [Q],X 3 [Q\) . 

Cooperative distributed source coding with two distortion criteria under reversal delivery and 
side information This concludes the proof of the converse and thus that of the theorem. 


E. Three encoders and three decoders subject to lossless/lossy reconstruction constraints with 
degraded side information 


Consider now the problem described in Fig. [ 6 ] where encoder 1 wishes to communicate the 
lossless the source X'j to nodes 2 and 3 while encoder 2 wishes to send a lossy description 
of its source AT,' to node 3 with distortion constraints D 32 and encoder 3 wishes to send a 
lossy description of its source X 3 to node 2 with distortion constraints D- 2 3 . In addition to this, 
the encoders perfom the communication using K communication rounds. This problem can be 


seen as a generalization of the settings previously investigated in [10]. This setting can model 
a problem in which node 1 generate a process X 3 . This process physically propagates to the 
locations where nodes 2 and 3 are. These nodes measure A " 2 and A 3 respectively. If node 2 is 
closer to node 1 than node 3, we can assume that X\ -e- X 2 -e- X 3 . Nodes 2 and 3 then interact 
between them and with node 1, in order to reconstruct X\ in lossless fashion and X 2 and A 3 
with some distortion level. 
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(X? 3 ,D 23 ) X&mX 



Figure 6: Three encoders and three decoders subject to lossless/lossy reconstruction constraints 
with degraded side information. 


Theorem 6 : The rate-distortion region of the setting described in Fig. [6] where X 1 -e-X 2 -e-X 3 
form a Markov chain is given by the union over all joint probability measures Px 1 x 2 x 3 u 3 ^ 2 {1 . K] u 2 ^ 3 {1 . K] 
satisfying the Markov chains 


(^1> X-2, V[23,Z,2]) X 3 , 
£4-s>2,« (Xi, X 3 , V[23,Z,3]) ~ e ~ X 2 , 

VZ € [1 : K], and such that there exist reconstruction mappings: 

023 {Xl, X 2 , V[23,_fr+1,2]) = -^23 With E d(X 3 , X 23 ) 
932 (Xi, X 3 , V[ 23 ,ir+i, 2 ]) = ^"32 with E d(A" 2 , X 32 ) 

of the set of all tuples satisfying: 


< D 23 

< D32 


(277) 

(278) 


Ri > H(X 1 \X 2 ) , 

(279) 

R 2 > -f( V[23,ir+i,2]! X 2 \XiX 3 ) , 

(280) 

i?3 > /(V[23,zr + i,2];X 3 |XiX2) , 

(281) 

Ri + R 2 > H(x 1 \X 3 ) + I(V [2 3,k+ i, 2 ]\X 2 \X 1 X 3 ) , 

(282) 
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The auxiliary random variables have cardinality bounds: 

{-i 

||« 2 —> 3 ,HI < Pillll^lin 11^1111^2,11 + 1 , le[l:K\ ( 283 ) 

i— 1 

Z-l 

||W 3 -2,t|| < 11^1111^31111^2^11 n 11^3,411^2,4 + 1, l e [1 : A'] (284) 

*=i 

Remark 15: Theorem [6] shows that several exchanges between nodes 2 and 3 can be helpful. 
Node 1 transmit only once at the beginning its full source. 

Proof: The direct part of the proof follows according to Theorem [I] by choosing: 

Ui-^3 t i = Ui^.2,1 = = U 2 ->ij = Z/ 3 _> 12,i = 0 VZ G [1 : A]: 

U 1 ^ 23,1 — U 2 =>.i 3 ,i — 0, VZ G [2 : K\ 

and Z7 i^23,i = Aa^is.i = AT The remanding auxiliary random variables satisfy VZ G [1 : K]: 

U 2 ^3,i -e- (X t , X 2 , V [23 ,i,2]) -e- * 3 , (285) 

U 3 ^ 2,1 -e_ (Aii, AT V[2 3 ,z, 3 ] ) -e_ AT 2 . (286) 

If a pair of tuple (Ai,A 2 ,A 3 ) and distortions (A> 23 ,-D 32 ) are admissible for the A'-steps 
interactive cooperative distributed source coding setting described in Fig. [6j then for all e > 0 
there exists no(e,K), such that Vn > no(e,K) there exists a A'-steps interactive source code 
(n, AT, 5F, Q) with intermediate rates satisfying: 

1 X 

-^logllJ'/H <Ri+e , i€ {1,2,3} (287) 

1 i=i 

and with average per-letter distortions with respect to the source 2 and perfect reconstruction 
with respect to the source 1 at all nodes: 


E 

d(X%,X ») 

< D 32 + £ , 

(288) 

Pr 

(*i T * 2 ") 

< £ , 

(289) 

E 

'd(X2,x&j 

< D 23 + £ , 

(290) 

Pr 

(at + xi^ 

<£ , 

(291) 
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where 


yn — I tj\y'-K\ rr[L'-K\ ^[1 :I\] 

-^-32 ~ 932 [Jl ,02 ,03 , 


■\rn 

^3 


X 2 " 3 = g 2 3 ( J? :K] , J\ l '- K] , jt K] , X 2 ) , (292) 


yn 1 q-\y.K\ rr[ l.K\ ^-[ 1 :A] yn 

A 31 — 531 I O 1 , O 2 ,03 , A 3 


yn 1 rj\\ '-K\ y[\-.K\ ^[1 :K\ yn 

^21— 921 \J\ ,02 ,0 3 ,^2 


. (293) 


For each t G (1,... ,n} and l G [1 : K], we define random variables t/ 2 —^ 3 ,z,[t] and U s ^ 2 ,i,[t] 
as follows: 


^2->-3,l,[t] — - {Ji, J 2 i X l[l-t-l}, A' 1 [ t+1:n ], X 2 [t + l:n], ^ 3 [ 1: t-i]) , 

U^3,i,[t} = (JIJ 2 ), le[2:K] 

U^2,i,[t] = JL le[l:K}. 


(294) 

(295) 

(296) 


These auxiliary random variables satisfy the Markov conditions (285) and (286), which can 
be verified from Lemma [TT| in the appendices. By the conditions ( |291[ ) and ( |289| ), and Fano’s 


inequality [211, we have 


H(X';\y i ) < Pr (a'“ + Aj) logjdKII - 1) + H, (pr(xr + A?,)) 4 ne„ , (297) 
//'A7'|A'" j < Pr (AT + AJ,j log 2 (||AT|| - 1) + //, (pr(A” # A” j] = ne n , (298) 


where e n (e) —> 0 provided that £ —> 0 and n —> oo. 
/) ifafe at node 1: For the first rate, we have 


n(i*i + e) > H ((299) 

> H [j\ UK] \X^X r ^j (300) 

= H (jr ] jt K] jt K] \X^) (301) 

= / (j} 1 :K] jt K] jt Kl , x ?\ x 2 x s) (302) 

> ntf (Xy | X 2 X 3 ) - H (x?\ X2j} 1:K] jt K] jt Kl ) (303) 

> nH{X l \X 2 X 3 ) - H(X?\Xd 2l ) (304) 

>n[H(X 1 \X 2 X 3 )-e n ] , (305) 

— n [H(Xi\X 2 ) — e n ] , (306) 


where 
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step (a) follows from the fact that by definition of the code the sequence J^' K \ j \'' A ' are 
functions of %,Xg), 

step ( b ) follows from the fact that by definition of the code the sequences J 2 ' K \ 

are functions of the sources(X", X%, X%), 


step (c) follows from the code assumption in fl293| ) that guarantees the existence of a 
reconstruction function X'" n = g 2l J^' K \ J^' K \X%\ 


• step (d) follows from Fano’s inequality in ( |298| ), 

• step (e) follows from the assumption that X\ -e- X 2 -< 
2) Rate at nodes 2 and 3: For the second rate, we have 


X-> form a Markov chain. 


n.(f?2 + £■) 2 H \dJ 2 


r[l:K] 


= I (J 2 ' K] \ x^x2 x^ 


(ft) 


> i [jt Kl ,xrj\x?x; 


(c) 


— T I 'jd-X n-d'- K \ st\ 3'-K]. yn yn yn 

— t ^ 2 /j J-2 J 3 2 1^1^3 

n 

X) SX r ( ^ 1:R } rrd-.K] rr[l:K\_ y | yn y 

~ 1^1 X2 2/3 , -*2[t] | A 1 ^2[t+l:n] 


(e) 


t= 1 
n 


— (y[23,iT+l,2][t];^2[t]|^l[t]^3[t]) 


(/) 


i=l 

n 


Y, I (V[23,X+l,2][Q];^2[Q]|A"i[Q]2f 3 [Q],(5 - f) 


t=l 


(9) 


> nl V[23,ir + i,2];X2|AiX 3 , 


(307) 

(308) 

(309) 

(310) 

(311) 

(312) 

(313) 

(314) 


where 


step (a) follows from the fact that j\ 1 ' A ' is a function of the sources (X\ l , X 2 . A"."), 
step ( b ) follows from the non-negativity of mutual information, 

step (c) follows from the fact that ( A "' ,J 2 ' ^ ^) are functions of the sources 

( V n V n Vn\ 

1^1 ? ^-2 5 ^-3 /’ 

step (d) follows from the chain rule for conditional mutual information, 


step (e) follows from the definitions (J294J), ( |295| ) and ( |296| ), the memoryless property of 
the sources and the non-negativity of mutual information, 
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• step (/) follows from the use of a time sharing random variable Q uniformly distributed 

over the set {1 ., n}, 

• step (g) follows by letting a new random variable V[ 23 , K+ i, 2 ] = (V[ 2 3 ,a:+i, 2 ][Q], Q)- 
By following similar steps, it is not difficult to check that 

n 

n(R3 + s) > (V[ 23) K+i, 2 ][t] : i^ 3 [t]|^ 1 [t]^2[t]) (315) 

t .=i 
n 

= (V[23,a:+i,2][Q]; X 3 [Q] \X l [Q]X 2 [Q], Q = t ) (316) 

t =i 

> nl (V[23 ,k+i, 2]] X 3 \XiX 2 S J . (317) 


3) Sum-rate of nodes 1 and 2: For the sum-rate, we have 
n(R\ + R 2 + 2e)> H + H (318) 

> 77 (319) 

= 7 (j\ VK] J 2 1:K] ; X['X; i X^ (320) 

> I (j} 1:K] jt K] -, A7X 2 n |X 3 n ) (321) 

= I (ji ' K] J 2 :K] ] X\ l \X^ + I ( y ji V - K] jl ] '- K] ji V ' K] ; XfXl l X!^ (322) 

= h (Xfxr:;) - h (x?\j} 1 :K] ji 1 :K] x^ 

+7 [j\ UK] J\’ :K] J\ X '' K] \ X^ | X\ l X!^j (323) 

> H (XfXf) - H(XfXf) + I (jl UK] jt K] jt K] ] XfX\ l X!^ (324) 


> n [77 (X^Ys) - e n ] + I j£ :K] f K \x^X^ (325) 

(e) n 

> Y. 1 (^’l 1:A " ] ^2 1:E " 1 ^’3 1:i:i:1 ^l[l:t-l]^ 1 [i+l:n]^ 3 [l:i- 1] ^2[t+l:n]; X 2[t] \X 1[t] X 3[t] ) 

t= 1 

+ n[H(X 1 \X 3 )-e n \ (326) 

n 

= n [H (A^lXs) - e n ] + ^ 7 (V [23 ,^+i, 2 ][t]; X 2W |X 1[t] X 3W ) (327) 

t=i 

= n [77 (X 1 \X 3 ) - e n + I (V [23>jRr+1 , 2][Q] ; X 2[Q] \X 1[Q] X m ,Q)} (328) 

= n [H (X,\X 3 ) - e n + I (v [2 3 ,k+i, 2] ; * 2 | ] , (329) 


where 
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step (a) follows from the fact that j \ 1 ' A ■ and j \ 1 ' A ^ are functions of the sources (X} 1 , X%, X7), 
step ( b ) follows non-negativity of mutual information, 


step (c) follows from the code assumption in ( |293| ) that guarantees the existence of recon¬ 


struction function X” = g^ljf \X. 


r n 

L 3 /’ 


• step (d) follows from Fano’s inequality in ( |291 [ ), 

• step (e) follows from the chain rule of conditional mutual information and the memoryless 
property across time of the sources (X”, X%, AX), and non-negativity of mutual information, 

• step (/) from follows from the definitions ( |294| ) and ( |295[ ), 

• step (g) follows from the use of a time sharing random variable 0 uniformly distributed 
over the set {1,..., n}, 

• step (h) follows from V [23 , k +i, 2 ] = (V[ 23 ,ir+i, 2 ][Q], Q). 

4) Distortion at node 2: Node 2 reconstructs a lossy X^ 3 = g 23 J^' K \ X% 


For each t G {1,..., n}, define a function X 23 [ t ] as beging the t-th coordinate of this estimate: 

X m] (Vp 3|Jf+1 , 2][t]l X 2[t] ) = g m [j} 1:K] , jt K \ jt K] ,X 2 ) ■ (330) 

The component-wise mean distortion thus verifies 

D 23 + £>E \d (x 3 , g 23 (j} 1:K] , jt K \ jt K] , x 2 "))] (331) 

(332) 


d (A^3[t], X 23 [ t ] (V[23,K+ 1,2] [t], X 2 [ t ]) 


1 


n 


t =1 


E e 


t=i 


d I X 3 [Q], X 23 [Q] 


= E 
= E 

where we defined function X 23 by 


(V[23,K+1,2][Q\, X 2[q] ) 

d (X 3 [q],X 23 [q] (V[ 23)J c + i j2 ][q],X 2 [q]) 
d ( X 3 , A" 23 ( V[ 23j ft-+i ;2 ], X 2 


Q = t (333) 

(334) 

(335) 


A 23 ^V[ 23 ,k:+i, 2 ], X 2 J — X 23 (Q, V [23 ,a'+i,2][q],X 2[ q]) A X 23[q] (V[ 23) i<- + i )2 ][Q],X 2 [Q]) .(336) 

5) Distortion at node 3: Node 3 reconstructs a lossy description X% 2 = g 32 J 2 ' K \ 

J 3 ' K \ X 3 ^. For each t <G {1,..., n}, define a function X 32 [t\ as beging the t -th coordinate of 
this estimate: 


X m (V [23lK+im ,X 3[t] )±g 32[t] (j} 1:K] ,ji 1:K] ,ji 1:K] ,X ; 


(337) 
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The component-wise mean distortion thus verifies 

£32 + e > E [d (x 2 , g 32 (ji 1:K] , jt K \ jt K \^ 


- ~ X] E d faith {V[23,K+l,2][t\, X 3[t] ) 

1 t =1 

1 n 

= - ^2 E d ( K X 2[Q], X32[Q] (V[23,A-+1 i 2][0], ^3[Q]) 

= E d (X 2 [Q], X 32 [Q] (V[23,A'+l,2][Q],^3[Q]) 

= E d (^X 2 , X 32 ^V[23,ir+i,2]) X 3 

where we defined function X 32 by 

X 32 ^V[23,ftT+l,3] j X^ = X 32 (Q, V[23,K+1,3][Q],-^3[Q]) 
- ^32[Q] (V[ 23 ,ir+ 1 , 3 ][Q],^ 3 [Q]) 

This concludes the proof of the converse and thus that of the theorem. 


Q — t 


(338) 

(339) 

(340) 

(341) 

(342) 


(343) 


VI. Discussion 


A. Numerical example 


In order to obtain further insight into the gains obtained from cooperation, we consider the 
case of two encoders and one decoder subject to lossy/lossless reconstruction constraints without 
side information in which the sources are distributed according to: 


Px 1 x 2 (xi,x 2 ) = at {xi = 1} 


\f 7 txo\ 


exp 


Xn 


2 erf 


+ (1 - a)l {H = 0} exp (-||) • (344) 

This model yields a mixed between discrete and continuous components. We observe that X 3 
follows a Bernoulli distribution with parameter a e [0 : 1] while X 2 given X\ follows a Gaussian 
distribution with different variance according to the value of X\ e (0,1}. In this sense, X 2 
follows a Gaussian mixture distributioif^l 


"’Although the inner bound region in Theorem 1 is strictly valid for discrete sources with finite alphabets, the Gaussian 
distribution is sufficiently well-behaved to apply a uniform quantization procedure prior to the application of the results of 
Theorem 1. Then, a limiting argument using a sequence of decreasing quantization step-sizes will deliver the desired result. See 
chapter 3 in 1221. 
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The optimal rate-distortion region for this case was characterized in Theorem [2] and can be 
alternatively written as: 

n{D) = |J {(RuR 2 ) : Ri > H{ X 1 \X 2 ) , 

p£C 

R 2 >I(X 2] U\X 1 ) , 

Ri + R‘2 > H{Xf) + I(X 2 - U |X0 } , (345) 

where 

jC = {pu\x x x-2 '■ there exists (xi,u) >-)■ g(xi,u) such that E[d(X 2 , g{X^ U))] < D} . (346) 

The corresponding non-cooperative region for the same problem was characterized in [5J: 

R(D) = U {(R h R2) : Ri > H(Xi\U) , (347) 

p£C* 

R 2 >I{X 2 -U\X 1 ) , (348) 

Ri T 7?2 > H^) + I{X 2 - U\X^ , (349) 

where 

= {pu\x 2 '■ there exists (x\,u) >-)■ g(xi,u) such that E [d(X 2 , g(X 1 , (/))] < D j . (350) 


From the previous expressions, it is evident that the cooperative case offers some gains with 
respect to the non-cooperative setup. This is clearly evidenced from the lower limit in Ri and 
the fact that C* C C. We have the following result. 


Theorem 7 (Cooperative region for mixed discrete/continuous source): Assume the source dis¬ 
tribution is given by (344) and that, without loss of generality, erg < a\. The rate-distortion region 
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from Theorem [2] can be written as: 


Ri > 


a 


a/2/T (T q c 


exp 


a 


xs 


2°'o 


H 2 (g(x 2 ))dx 2 


a/27TC} J —c 


exp 


'2 

x 2 


2c? 


H 2 (g(x 2 ))dx 2 , 


Ri > 


1 1 I 

2 log 


2(1—a) 2a 


C 


1 




cc 

2 


log 


acii 


D — (1 — a)c. 


o / J 


D<al 


D>al 


R\ + R 2 > < 


1 /c 2 ^ 1 Q V 2 “' 

ff 2 (a) + -log 0 — 1 


a 


#»(<*) + - 


log 


aa r 


D <°o 


D>al 


D — (1 — a)c 2 

where Tf 2 (£) = — - logz — (1 — 2 ) log (1 — z) for z G [0,1], [x] + = max {0, x} and 


a 


g(x 2 ) = 


a/^TCx 


exp 


xs 


2c 2 


a 


exp 


Xn 


+ 


1 — a 


exp 


Xn 


2c 0 2 


A/^cr^V 2 c 2 7 ^0 

Proof: The converse proof is straightforward by observing that when I) < of. 

I(X 2 ; U\Xf) = h{X 2 \Xf) - h(X 2 \U, X 0 > h(X 2 \X 1 ) - l - log (2 tt eD) 

and h(X 2 \Xi) = f log (27rec 2 ) + log (27reCo). For the case when Cq < D < ac 2 + (1 ■ 
we can write: 


J(X i; Z7|X0 = h{X 2 \X 1 ) - h(X 2 \U,Xi) 

> h{X 2 \Xj) - ah(X 2 \U, X\ = 1) - (1 - a)h(X 2 \Xi = 0) 

a , . 9 . , /27t e(D — (1 — a)c?) 

= - log (27rec 2 ) - a log -^^ 

2 \ a 

cm\ 


2 l0g \D-(l-a)a { 


When D > aa\ + (1 — a)a^ we can lower bound the mutual information by zero. 
The achievability follows from the choice: 

X U if X, = 0 
,U if X, = 1 , 


g(u,x 1 ) = 


°0 

CT o+ CT i 0 
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U = 


( 356 ) 


and by setting the auxiliary random variable: 

X 2 + Zq if X] = 0 , 

X 2 + Z 1 if X x = l, 

where Z (i . Z\ are zero-mean Gaussian random variables, independent from X 2 and X 1 and with 
variances given by: 

- -> - ^ (357) 




°ii - 


Da\ 


' 0 -~ a\-D ’ 

for D < (Tq while for a% < D < aaf + (1 — a)a q, we choose: 

i D ~ (1 - «Vo] <?i 


°io 00 > a Ii - 


(358) 


acrf — [D — (1 — a)a^] 

Finally, for D > aaf + (1 — a)cro, we let cr| —> 00 and a 2 Zl —> 00 . ■ 

Unfortunately, the non-cooperative region is hard to evaluate for the assumed source modep] 
In order to present some comparison between the cooperative and non-cooperative case let us 
fix the same value for the rate B\ in both cases and compare the rate R- 2 that can be obtained 
in each case. Clearly, in this way, we are not taking into account the gain in R\ that could be 
obtained by the cooperative scheme (as H(X 1 \X 2 ) < H{X±\U) for every U -e- X 2 -e- X{). For 
both schemes, it follows that for fixed R ,\: 


R 2 > max {I{X 2 - U\XJ, H{ X x ) + I(X 2 , U\X 1 ) - R^ . 


(359) 


From Theorem [7} we can compute (359) for the cooperative case. For the non-cooperative case 
we need to obtain a lower bound on /( X 2 ; U |Xi) for pu\x x G CZ■ It is easy to check that: 


I{X 2 -U\X,)> 


a 


Z 0 
A) 


where 


log (ft)' 

(360) 

= 0]) 2 |A' 1 = o] , 

(361) 

i]) 2 !*! = 1 • 

(362) 


The distortion constraint imposes the condition: 

(1 — a)/3o + a/3i < D 


(363) 


"However, there are cases where an exact characterization is possible. This is the case, for example, when X\ and X 2 are 
the input and output of binary channel with crossover probability a and the distortion function is the Hamming distance |5j. 


January 19, 2016 


DRAFT 











57 


In order to guarantee that (360) and (361) are achievable, under the Markov constraint U 
X 2 -e- Xi, the following conditions on Pu\x 2 (u\x 2 ) should be satisfied: 


Pu\x 2 (u\x 2 ) 


1 


a/27T(J0 


exp 


Xn 


2°o 


and 


SZoPmM**) 


Pu\x 2 (u\x 2 ) 


a/27T(J0 


exp 


t .2 


2a 0 2 




v / 27p9o 


exp 


(^2 ~ /o(w))" 
2/3 0 


(364) 




exp 


/v .2 

x 2 


2 cr 2 


X^ 0 Pt/|X 2 (M|x2) 


v / 27ro' 1 


exp 


~2 

x 2 


2cr 2 


dx-2 


y/2nfh 


exp 


(x 2 - fi(u)Y 
2ft 


(365) 


where 


fo(U) = E X2luXl [X 2 \U,X 1 = 0] , h(U) = ¥, X2WXl [X 2 \U,X 1 = 1] 


(366) 


The characterization of all distributions Pu\x 2 {u\x 2 ) that satisfies (365) and (366) appears to be 
a difficult problem. In order to show a numerical example, we shall simply assume that: 


Pu\x 2 (u\x 2 ) = 


\Z2n< 


exp 


7T(J„ 


('u - x 2 y 

2cr 2 


(367) 


Indeed, this choice satisfies simultaneously expressions (365) and (366). In this way, we can 
calculate the corresponding values of /3 0 and ft obtaining the parametrization of I(X 2 : U\X \) 
as function of cr 2 : 


I(X 2 ]U IX,) = 

with the following constraint: 


1 /J i i 


a 


log 


°n + < 


(!-«)■ 




erf, 


+ OL m 


a 

+ 2 l0g 


vi + Vu 


erf 


<4°1 


= D . 


(368) 


(369) 


°o + < <ri + < 

We can replace ( |369 ) in (359) to obtain an indication of the performance of the non-cooperative 
case when R\ is fixed. 


We present now some numerical evaluations. As equation (359) is valid for both the coop¬ 
erative and the non-cooperative setups, it is sufficient to compare the mutual information term 
I(X 2 ; U\X{) for each of them. Let us consider the next scenarios: 

1) a = 0.1, ctq = 0.01, a 2 = 2, 

2) a = 0.1, (Tq = 0.5, erf = 2. 
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Figure 7: Comparison between the cooperative and the non-cooperative schemes. 


From Fig. [ 7 ] we see that in the case ctq <C <j\ the gain of the cooperative scheme is pretty 
noticeable. However, as <j\ becomes comparable to af the gains are reduced. This was expected 
from the fact that as ag —> erf, the random variable X 2 converges to a Gaussian distribution. 
In that case, the reconstruction of X 2 at Node 3 is equivalent for the cooperative scenario to 
a lossy source coding problem with side information X\ at both the encoder and the decoder 
while for the non-cooperative setting to the standard Wyner-Ziv problem. It is known that in this 
case there is no gains that can be expected 0. 

B. Interactive Lossless Source Coding 

Consider now the problem described in Fig. [8] where encoder 1 wishes to communicate lossless 
the source X” to two decoders which observe sources X'd and X". At the same time node 1 
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wishes to recover X% and A(” lossless. Similarly the other encoders want to communicate lossless 
their sources and recover the sources from the rest. It is wanted to do this through K rounds of 
exchanges. 


*2" ~ *l" *23 ~ *3" 



Theorem 8 (Interactive lossless source coding): The rate region of the setting described in 
Fig. [8] is given by the set of all tuples satisfying: 


Ri>H{X 1 \X 2 X 3 ) , 

(370) 

R 2 >H(X 2 \X 1 X 3 ) , 

(371) 

R 3 >H(X 3 \X l X 2 ) , 

C372) 

Ri + R2>h(x 1 X‘ 2 \x 3 ) , 

(373) 

Ri + R^H^XslX^ , 

(374) 

R 2 + R 3 >H(X 2 X 3 \X 1 ) . 

(375) 


Remark 16: It is worth observing that the multiple exchanges of descriptions between all 
nodes cannot increase the rate compared to standard Slepian-Wolf coding 0 

Proof: The achievability part is a standard exercise. The converse proof is straightforward 
from cut-set arguments. For these reasons both proofs are ommited. ■ 
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We should note that for this important case, Theorem |T| does not provide the optimal rate- 
region. That is, the coding scheme used is not optimal for this case. In fact, from Theorem |T] 
and for this problem we can obtain the following achievable rcgiorf^j 


Ri > H(X i|X 2 ) , 

(376) 

R 2 > H{X 2 \X 1 X 3 ) , 

(377) 

R 3 > H(XslX.X,) , 

(378) 

+ R 2 > H{X x X 2 1X 3 ) , 

(379) 

+ R 3 > H(X 2 X 3 \X 1 ) . 

(380) 


It is easily seen that in this region the node 2 is not performing joint decoding of the descriptions 
generated at node 1 and 3. Because of the encoding ordering assumed (1 —2 —3) and the 
fact that the common description generated in node 2 should be conditionally generated on the 
common description generated at node 1, node 2 has to recover first this common description 
first. At the end, it recovers the common description generated at node 3. On the other hand, 
nodes 1 and 3 perform joint decoding of the common information generated at nodes 2 and 
3, and at nodes 1 and 2, respectively. Clearly, this is consequence of the sequential encoding 
and decoding structure imposed between the nodes in the network and which is the basis of the 
interaction. 

If all nodes would be allowed to perform a joint decoding procedure in order to recover all the 
exchanged descriptions only at the end of each round, this problem would not appear. However, 
this would destroy the sequential encoding-decoding structure assumed in the paper which seems 
to be optimal in other situations. 


VII. Summary 

The three-node multiterminal lossy source coding problem was investigated. This problem 
is not a straightforward generalization of the original problem posed by Kaspi in 1985. As 
this general problem encompasses several open problems in multiterminal rate distortion the 

12 Consider t/i_> 23 ,i = Xi, U 2 ->i 3 ,i = X 2 and U 3 ->i 2 ,i = X 3 and the other auxiliary random variables to be constants for 
all l€[ 1 : K}. 
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mathematical complexity of it is formidable. For that reason we only provided a general inner 
bound for the rate distortion region. It is shown that this (rather involved) inner bound contains 
several rate-distortion regions of some relevant source coding settings. It In this way, besides 
the non-trivial extension of the interactive two terminal problem, our results can be seen as a 
generalization and hence unification of several previous works in the field. We also showed, 
that our inner bound provides definite answers to special cases of the general problem. It was 
shown that in some cases the cooperation induced by the interaction can be helpful while in 
others not. It is clear that further study is needed on the topic of multiple terminal cooperative 
coding, including a proper generalization to larger networks and to the problem of interactively 
estimating arbitrary functions of the sources sensed at the nodes. 
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Appendix A 

Strongly typical sequences and related results 

In this appendix we introduce standard notions in information theory but suited for the 
mathematical developments and proof needed in this work. The results presents can be easily 


derived from the standard formulations provided in |22| and [231. Be X and y finite alphabets 
and ( x n ,y n ) G X n x y n . With V(X x 30 we denote the set of all probability distributions on 
X x y. We define the strongly 5-typical sets as: 

Definition 3 (Strongly typical set): Consider p G V(X) and 5 > 0. We say that x n G X n is 
p5- strongly typical if x n G 7Jj* with: 

5 


'W 


\x n G rT n : 

iV(a|a: n ) , . 

p(a) 

l 

n 


< 


\x\ 


Va G X such that p(a) f 0 


(381) 


where N(a\x n ) denotes de number of occurrences of a G X in x n and p G V(X). When 
X ~ Px {'■)') we can denote the corresponding set of strongly typical sequences as 
Similarly, given p X v G V (X x y) we can construct the set of ^-jointly typical sequences as: 


T n 

>[XY]S 


= { (, x n ,y n ) G X n x y n : 


N(a, b\x n , y n ) 


n 


Pxv(a,b ) 


< 


5 


11 ^ 1111 ^ 11 ’ 


V(a, b) e X x y such that pxv (a, b) ^ 0} . 


(382) 


We also define the conditional typical sequences. In precise terms, given x n G X n we consider 
the set: 


r^ x]5 (x n ) = \y n ey 


N(a, b\x n , y r 


n 


- pxv (a, b) 


< 


11 ^ 1111 ^ 11 ’ 


V(a, b) G X x y such that Pxy(o,, b) 0 f . 

Notice that we the following is an alternative writing of this set: 


(383) 


7j?|x] S (*") = {»" 6 V” : K. if) 6 75- r|s } . (384) 

We have several useful and standard Lemmas, which will be presented without proof (except 
for the last one): 

Lemma 1 (Properties of typical sets /f23]/); The following are true: 

1) Consider ( x n ,y n ) G 7jJ y]e . Then, x n G 7|' y]e , y n G 7[y ]o G T^ Y]e (y n ) and y n G 

’[Y |X]ef X f • 
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2) Be T- ]x]e (x n ) with x" $ 7jJ ]e . Then Tf Y]x]e (x n ) = 0 . 

3) Be (. X n ,Y n ) ~ Ut=iPxY(x t ,y t ). If x 11 G T^ x]f we have 

2 -n(H(X) +l 5(6)) < Px J x n ) < 2-«(»W-*(«)) 


with 5(e) —>■ 0 when e —>• 0. Similarly, if y n G 7[y| Y ] e (:r n ): 

2 _ n( H(y|x) + 5'( e) ) < pyn|x „(^|^) < 2 -«(*(m)-«'W) 


with 5'(e) —>• 0 when e —>■ 0 . 

Lemma 2 (Conditional typicality lemma [\23\j): Consider dc product measure nT=i Pxy{xt, ?//)• 
Using that measure, we have the following 

Pr{7ft € }>l-o(cr“ /<e >), c>l 

where /(e) —> 0 when e — >• 0. In addition, for every x n G T$ ]e , with e' < -pii we have: 

Pr {Tv\ X] ^ n )K} >1-0 (cr’’"' 1 ) , Cj > 1 


where y(e, e') — > 0 when e, e' — > 0. 

Lemma 3 (Size of typical sets /[23]/j: Given p Y y G P (X x y) we have 
-||7jS],|| < H(A') + 5(e), i||7fj |t || > ff(A) - i'ft.n) 

71 1 1 71 1 1 

where 5(e), 5'(e, n) —* 0 when e -> 0 and n —> oo. Similarly for every G A n we have: 


i|| 7 ;? We (x”)|| < H(y|A) + «s( £ ) 

with 5(e) — > 0 with e — )■ 0. In addition, for every x n G T{fy With e' < pjjj we have: 

/|7i? We ( I ”)ll >H(Y\X)-S(eX,n) 


where 5'(e, e', n) —* 0 when e,e'G0 and n —* oo. 

Lemma 4 (Joint typicality lemma /[23]/): Consider (A", Y, Z) rs_/ PxYz(x,y,z ) and (x n ,y n ) G 

there exists 5 / (e,e / ,e",n) 


7j'/y] e , with e' < pu and e < e". If p Z "|X’ 

which goes to zero as e, e', e" —* 0 and ra —* oo and: 




2-n(/(y;Z|X)+<5') 


< Pr < Z™ G 


T Tl ('T 11 1 I T 
'[Z\XY]e\ X id 
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Lemma 5 (Covering Lemma [22]): Be (U. V, X) ~ Puvx and (x n . u n ) G T\xupn e ' < jpy 
and e < e". Consider also {V n (rn)} 2 m=1 random vectors which are independently generated 
according to \ l7 - 11 l e . Then: 

Pr \y n (m) £ T { v ]ux]e (x n ,u n ) for all m} -> 0 

L 1 J n—>oo 

uniformly for every (x n , u n ) G 'T xu y if: 

R > I(V\X\U) + 5(e,e',e",n) 

where 5(e, e', e", n) —>• 0 when e, e', e" — > 0 and n — * oo. 

Corollary 1: Assume the conditions in Lemma [5] and also: 


(385) 


(386) 


Then: 


Pr{(A",t/")e7j5„,,,}->1 


Pr {(£/”, A'”, y"(m))) $ 1$ xv] , for all m} -> 0 


(387) 


(388) 


when (|386|) is satisfied. 


Lemma 6 (Packing Lemma [22]): Be (U1U2WV1V2X) ~ Pu { u 2 wv^v 2 x, (x n , w n , v™, v%) G 
TjxwiVale' and e ' < prfaj and 6 < min{ei,e 2 }. Consider random vectors {^(mi)}^ 
and (f/ 2 (m 2 )})^. 2 = i which are independently generated according to 


1 


> * — 1)2, 


||%|v i w]f l (w ri ,p n )|| 

and Al,* 4 2 are positive random variables independent of everything else. Then 

Pr {(Ui(mi), £7 2 (m 2 )) e ^ lC/l | XWl y 2]e (a; n ,w ri ,<,'U 2 ) for some (mi,m 2 )} 0 (389) 

uniformly for every (x n , w n , i>", n 2 ) G 7jJ, Wi v - 2 ] e , provided that: 


logE [A A] 


n 


< I (Up XV 2 U 2 \WV l ) + / (Up XV 1 U 1 \WV 2 ) - I (Up U 2 \XWV 1 V 2 ) - 8 (390) 


where 8 = 8(e, e', e 1} e 2 , n) —>• 0 when e, e', ei, e 2 —>■ 0 and n —> 00 . 

Corollary 2: Assume the conditions in Lemma [6j and also: 


Pr {(A”, W\ Vr, 6 7JV,,, -> 1 


(391) 
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Then: 


Pr { U;(m 2 ), X ", TV", V?, 14”)) 6 Tfttr.j™,*,,* for some (mi,m 2 )} -» 0 


( 392 ) 


when ( |390[ ) is satisfied. 

Lemma 7 (Generalized Markov Lemma / [2?] / ): Consider a pmf puxy belonging to Y (Y x y x U) 
and that satisfies de following: 

Y -e- X -e- U 


Consider ( x n ,y n ) G , and random vectors U n generated according to: 


Pr i U n = u n 


x\y\U" eT? um An) 




I T n 


' [U\X]e"( Xn )W 

For sufficiently small e,d,e" the following holds uniformly for every (x n ,y n ) G 7r£yi e ,: 


(393) 


Pr 




z n ,Z/ n 


,l r "e^ m ..,(x”)} = 0(c-”) 


(394) 


where c > 1. 

Corollary 3: Assume the conditions in Lemma |7j and also: 


Pr { (X n , Y n ) e Xx Y \e’ } ->1 

1 J n^-oo 

(395) 

and that uniformly for every (x n . y n ) G T^yy- 



Pr{F"^7i5 We „(i”) 

x n ,y n } -> 0 

J n — ^oo 

(396) 

we obtain: 



Pr{(F",x”,r”) e ySxYj,} -»i. 

L J n—>• oo 

(397) 


Lemma [7] and Corollary [3] will be central for us. They will guarantee the joint typicality of 
the descriptions generated in different encoders considering the pmf of the chosen descriptions 
induced by the coding scheme used. The original proof of this result is given in [4] and involves a 
combination of rather sophisticated algebraic and combinatorial arguments over finite alphabets. 
Alternative proof was also provided in |22|, 


Pr {(U n X n ,Y n ) e 'Jp XY] y ->1 

L J n—> oo 


(398) 
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which strongly relies on a rather obscure result by Uhlmann [25] on combinatorics. In [24] a 
short and more general proof of this result is given. 

We next present a result which will be useful for proving Theorem |T[ In order to use the 
Markov Lemma we need to show that the descriptions induced by the encoding procedure in 
each node satisfies ( |393[ ). 

Lemma 8 (Encoding induced distribution): Consider a pmf puxw belonging to V (U x X x W) 
and e' > e. Be {U n {m)} m=1 random vectors independently generated according to 


\\T[U\W}e'{w n )\\ 

and where (W n , X n ) are generated with an arbitrary distribution. Once these vectors are gener¬ 
ated, and given x n and w n , we choose one of them if: 


(u n (m),w n ,x n ) G Tfi WX ]ei for some m G [1 : S] . (399) 

If there are various vectors u n that satisfies this we choose the one with smallest index. If there 
are none we choose an arbitrary one. Let M denote the index chosen. Then we have that: 

1 {«" e TpixH'i.X".™”) 


Proof: From the selection procedure for M we know that: 


\T[u\xw]c{x n ,w r 


. (400) 


M = f{ 1 {(U n (m),X n ,W n ) G T$ xw]e },m G [1 : S}) , (401) 

where /(•) is an appropriate function. Moreover, because of this and the way in which the 
random vectors U n are generated we have: 


U n (M) -e- (1 {(U n (M),X n , IL n ) G Tfi xw]e } , W n ,X n ) -e- M 


(402) 


We can write: 


Pr [U n (M) = u n x n , w n , U n (M) G T^ xw]t (xf w n ) | = 

V Pr [M = m\x n , to”, U n (m) e Tfo xw)t (x n , to”)} x 

X,to", U n (m) e Xy\ xw] ,(x n , to"), M = m\. 


?7I=1 

Pr { U n (m) = u 


(403) 
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From ( |402[ ), the second probability term in the RHS of ( |403[ ) can be written as: 

Pr W n (m) = u n 


x n , w n , U n (m) E 7$ ]xw]e (x n ,w n )j Vm . 

We are going to analyze this term. It is clear that we can write: 

Pr | U n (m) = u n , U n (m) 6 T^ xw]t (x\w’‘) {«” 6 7 ^ lwx] Jx",w")} 


(404) 


xPr < U n (m) = u r ‘ 


x n , w n \ Vm (405) 


This means that 


Pr{(7”(m) = e 


>1 

“"e7[g |H , x] ,(i", ro »)n7[g |w|e ,( w ”)' 


Pr < 

U"(m)eT { Z lxw]e (x",w") 

X n , W n 

• 1 \7[u\w\ 

f /(w ri )|| 


Vm . 


(406) 


From (|406|) and the fact that for e' > e, we have that 7^ yw ,] £ (i“, w") C T^ W]e ,(w n ) we obtain: 


'[l7|W]e' 


Pr \U n {m) = u n x n , w n , U n {m) G Tfi ]xw]£ (x n , w n ) \ = 


1 {f £ 

\\T[U\ X W]e(x n ,W n )\\ 


Vm . 

(407) 


From this equation and (403) we easily obtain the desired result. 

We present, without proof, a useful result about reconstruction functions for lossy source 
coding problems: 

Lemma 9 (Reconstruction functions for degraded random variables /[/3|/): Consider random 
variables (X,Y,Z) such that X -e- Y -e- Z. Consider an arbitrary function X = f (Y. Z ) 
and an arbitrary positive distortion function d(-, •). Then 3 g*(Y) such that 

E[d(X,g*(Y))] < E[d(XJ(Y,Z ))] . (408) 


Finally we present two lemmas about Markov chains induced by the interactive encoding 
schemes which will be relevant for the paper converse results 

Lemma 10 (Markov chains induced by interactive encoding of two nodes): Consider a set of 

n 

three sources (X n ,Y n , Z n ) ~ Yi PxYz(x t ,yt, z t ) and integer K e N. For each l e [1 : K] 

t =i 

consider arbitrary message sets X l x , X l y and arbitrary functions 


f l ( \ n 'ji 1 - 1 - 1 ] yd 1 ^- 1 ]) _ 

Jx \ y J O x 5 O y ) O x 1 

(409) 

fl (yn rr\\:l\ _ jl 

J y V ’ ^ x J ^ y ) 

(410) 
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with J x G X l x and J y G X l y . The following Markov chain relations are valid for each t G [1 : n] 
and l G [1 : K]\ 

1 ) (J x , X[ 1:t _i],y[ t+ i; n] ) -e- X[ t ] -e- {Y [t] ,z [t] ) , 

2) -e- (j± 1:l ~ 1] , ji 1:l - 1] ,X [1:t] ,Y [t+1:n] ) -e- (%%]) , 

3) {Jy, -e- -e- (X [t] ,Z [t] ) , 


4) X [i+ i:n] -e- [J^ :K \ji 1:K \x [1:t] ,Y [t+1:n] 


5) 

6) (j£ :Ki ,Jy 


q-[l-.K\ rM :K\ Y y 

Jx iJy 1 1 ]) I[t-.n\ 


- {Y\t],Z [t] ) , 
( x [t] ,Z [t] ) , 


[l\K] y y 7 7V 

i ^[t+l:n], ^[lii-lp -^[t+linp 1 [l:t— 1 ] 


{X[t \, Y[ t ]) -e- Z[ t ] 


Proof: Relations 1), 2) and 3) where obtained in [13]. For completeness we present here 
a short proof of 2). The proof of 1) and 3) are similar. For simplicity let us consider A = 


I ( J l x x \t+i-.n]',Y[t]Z[t] 

(a) 


Jx ' 1 1] Jy 1:l 1] X [1:t] Y [t+1:n] ). We can write the following: 


A < I [ J l x X [t+l . n y,Y [l:t] Z [t -\ 


S /1 x, 


^t+l-.n}]Y[ 1:t ]Z[i] 




= H X , 


L [t+l:n] 


WHH:r 




(c) 

< / X 




- (X [m:n 

- H (y M z K 


= H (Y [ht] Z lt] 

< J (x li+l! „i; jf'- 21 JJ 1: ‘- 21 A' [1;t] y„ +l! , 


(411) 


where 

• step (a) follows from non-negativity of mutual information, 

• step (b) follows from the fact that J x = f! r (^X n , J^ 1 ' 1 ^, Jy 1 ' 1 ^ 1 ^, 

• step (c) follows from the fact that J y l = f l y {\' n . Jx^^.J,' 1 2 j and that conditioning 
reduces entropy, 

• step (d) follows from the fact that j]r x = f l x ^X n , Jy 1J ~ 2 ^ and that conditioning 

reduces entropy. 

Continuing this procedure we obtain: 


A < I 


X[ t+ l:n]\Y[i.t\Z^ 




= 0. 


(412) 
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This shows that 2) is true. 4) and 5) are straightforward consequences of 2) and 3). Just consider 


J x +l = Jy +l = 0. The proof for 6) is straightforward from the fact that ( J r 'J 
only function of ( X n ,Y n ). 


[1:K] ^[1 :I<] 

y 


is 


Corollary 4: Consider the setting in Lemma 10 with the following modifications: 
. X -e- Z -e- Y , 

fl I VB ryn 1] rr\ 

* Jx l A 1 Z 1 Cx , Jy 

The following are true: 

1 ) {Jx , Z V-:t-l],Y[t+V.n}) Z[t] Y[t] 


r[l:i —1]^ _ q-l 

X 


[ 1 :/— 1 ] ^ 7 " [ 1 :/ 1 ] 


— q- 

sJx j %Jl 


) ^[l:t]) ^ jt+l:n] 


2) («/j. -< 

3) ^ 

4) (Z [t+1:n] , X n ) -e- ( jf* 1 , , Z [1:t] , Y[ t+1:n] 


iJy i * [t:n] 


(X M ,Z W ) , 
- ^ • 


Proof: The proof follows the same lines of Lemma 10 


We consider next some Markov chains that arises naturally when we have 3 interacting nodes 
and which will be needed for Theorem [6l 

Lemma 11 (Markov chains induced by interactive encoding of three nodes): Consider a set of 
three sources (. X n ,Y n ,Z n ) ~ II”=i PxYz(%t, Ut, zf) and integer K G N. For each l G [1 : K\ 
consider arbitrary message sets X l x , X l y , X[ and arbitrary functions 


rl ( yn rr[V.l-l) y[V.l-\}\ _ rrl 

J x V ’ ^ x 5 ^ y 1 z ) ^ x ? 

(413) 

rl (yn rr\\ :l\ 1] q-[l:l-\}\ _ jl 

Jy \ A 5 'J x 5 i z ) *^y ? 

(414) 

rl (y n rr[l:l\ _ jl 

J z v ’ u X 5 u y i z ) u z 

(415) 


with j! r G X l x , Jy G X 1 and j\ G X\. The following Markov chain relations are valid for each 
t G [1 : n] and l G [1 : K]: 

1) X [t+1:n] ,Y [t+1:n] , Z[i :t _i]) -e- (X [t] ,Y[t\) -e- Z [t] , 

2) {JLJy,Y[i :t -i]) -e- (^i 1:i_1] , ^{ 1:l_1] , ^J 1:i “ 1] , JSC", -e- , 

3) (J‘,z lt+1:n] ) -e- ji 1:J - 1] ,x w ,yi t+ i :n ],z [1:t] ) -e-y[ t] , 

4) Z [t+1:n] ^(jl 1:K] ,jj 1:K] ,jJ 1:K] ,XfY^ 1:n] ,Z [1:t] ) -e-Y[ t] , 

5) >11:^!] ♦ -8- ^ [t] . 

Proof: Along the same lines of Lcmmifi7)| and for that reason omitted. ■ 
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X? 


XX 


v : r 


V? 



f/f 

/ 72 - 


Figure 9: Cooperative Berger-Tung problem. 


Appendix B 

Cooperative Berger-Tung Problem with Side Information at the Decoder 

We derive an inner bound on the rate region of the setup described in Fig. [9j It should be 
emphasize that we will not consider distortion measures, we only focus is on the exchange 
of descriptions. Encoders 1 and 2 observe source sequences X" and X 2 , and also have ac¬ 
cess to a common side information V™. Whereas, the decoder has access to side informations 
(X 3 , 17”, l/ 2 n ). Upon observing X[ l and Vj”, Encoder 1 generates a message Mi which is 
transmitted to Encoder 2 and the decoder. Encoder 2, upon observing (X 2 , Vj”) and the message 
Mi, generates a message M 2 which is transmitted only to the decoder. Finally, the decoder 
uses messages (M 1 ; M 2 ) and the side informations (X"', Vj”, Vj") to reconstruct two sequences 
((/”, U 2 ) which are jointly typical with (X™, XXI, XX, Vj" , Vj” ). In precise terms, we will assume 
the following: 

• A probability mass function px 1 x 2 x 3 u 1 u 2 v 1 v 2 which takes values on cartesian product finite 
alphabets X, x X 2 x X. x 7/j x U 2 x Vj x V 2 , and that satisfies the following Markov chains: 

Ui -e- (Xi, Vi) -e- (X 2 , X 3 , V 2 ) , U 2 -e- (U 1; X 2 , U) -e- (X 1; X 3 , U 2 ) . (416) 

• Five random vectors (X”, X 2 , X%, V{ 1 , VJ 1 ) (not necessarily independently and identically 
distributed with px 1 x 2 x 3 ViV 2 which take values on alphabets X," x X 2 n x X 3 n x Vj" x V 2 
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such that, for every e > 0, 


Bm AJ.K-.l?) 6 7jJ lMW *} = 1 • (417) 

Definition 4 (Cooperative code): A code (n, /", f 2 , g n , Adi, Ad 2 ) for the setup in Fig. [9] is 
composed by: 

• Two set of indices Adi, M 2 . 

. An encoding function /” : X™ x V” —>• Adi, such that /”(a:”,n”) = mj. 

. An encoding function : X% x V™ x Adi —>■ Ad 2 , such that f 2 {x 2 -, n”, mi) = m 2 . 

• A decoding function g n : A^xVfx'ld? xAdixAd 2 -> U™xU%, such that g n (x^, v™, v 2 , mi, m 2 ) 

Definition 5 (Achievable rates): We say that (R 1: R 2 ) are e-achievable if there exists a code 
(n, A n , f 2 , g n ,M u M 2 ) such that: 

- log || Adi|| < Ri + e , - log || M 2 1 | < R 2 + e (418) 

n n 


and 


Pr 


-n jjn T/n T/n T/n T/n yn\ /7-71 1 ^ 

1 > ^2 i V 1 A 2 j i ' {U 1 U2V 1 V2X 1 X 2 X 3 \e f — 6 


(419) 


The closure of the set of all achievable rates (Ri,R 2 ) is denoted by IZcbt- 
The following theorem presents an inner bound to 7 Z C bt- 

Theorem 9: (Inner bound on the rate region of the cooperative Berger-Tung problem) Consider 
R^cbt 1 th e closure of the set of rates satisfying: 


Ri > I{Xi,Ui\X 2 Vi) , 

r 2 > i(x 2] u 2 \xyv 2 u x ), 

M + R2 > I(yXiX 2 ,UiU 2 \X 3 ViV 2 ) , 


where the union is over all probability distributions verifying ( |416| ). Then T^cbt^ — R-cbt- 
Remark 17: Notice that we are not asking for (A/, X%, AT, Vfi, Vfi) to be independently and 
identically distributed. This is in fact not needed for the result that follows. For us, when trying 
to use this result, the case of most interest will be when (A”, X 2 ,X^) is generated using the 
product measure Yl^ =1 Px 1 x 2 x 3 (xii,X 2 i,x 3 i), (that is, when (X l X 2 X :i ) is a DMS). However, 
(Vfi, V 2 ) will not be independently and identically distributed. Still, (|417[) will be satisfied. 
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Remark 18: Notice that unlike the classical rate-distortion problem we are not interested in 
an average per-symbol distortion constraints at the decoder. We only require that the obtained 
sequences be jointly typical with the sources. Clearly the problem can be slightly modified to 
consider the case in which reconstruction distortion constraints are of interest. In fact, case (C) 


reported in [26], considers a similar setting. Here, given the importance of this result for our 
interactive scheme, we present a slightly different and more direct proof of the achievability, 
where we discuss the key points in the encoding and decoding procedures which will be relevant 
for our extension to the interactive problem. 

Proof: Our proof uses standard ideas from multiterminal source coding. As Vj" is common 
to both encoders and decoder we can set without loss of generality Vf = 0. Conditioning with 
respect to Vj the final expressions can take into account the situation in which Vf f 0. 


A. Coding generation 

We randomly generate 2 nRl codewords Uf(k), k 6 [1 : 2 R1 1 according to 


um 


r\j 


1 



/ ~Tn 

1 [Ui]e cd 


Cd > o . 


(420) 


These 2 nRl codewords are distributed uniformly over 2 nRl bins denoted by B\ (m ), where 
mi e [1 : 2 nRl ]. For each codeword with k e [1 : 2 nRl ] , we randomly generate 2 nR2 

codewords according to: 


U?(l,k) 


U 2 e 


T, 


\U2\U1\e-cd 




i £cd 0 


(421) 


with l G [1 : 2 nR ' 2 ] . The 2 n ( Rl+ri d codewords generated are distributed uniformly in 2 ni?2 bins, 
denoted by £> 2 (m 2 ), m 2 G [1 : 2 nR ' 2 ]. It is worth to mention that the codewords {[/)'(/, k)} are 
not distributed in a different structure of bins for each k , but on only one super-bin structure of 
size 2 n( ' Rl+R ^/2 nR2 where £I 2 (m 2 ) does not needed to be indexed with k. 

As will be clear, this will not constraint the decoder to use successive decoding and instead 
use joint decoding in order to recover the desired codewords (£/[', Uf). Finally all codebooks 
are revealed to all parties. 
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B. Encoding at node 1 

Given x”, the encoder search for k e [1 : 2 ni?1 ] in such a way that: 

W. «?(*))£ 7jJ lt;ijQ , €2 > 0 . (422) 

If more than one index satisfies this condition, then we choose the one with the smallest index. 
Otherwise, if no such index exists, we choose an arbitrary one and declare an error. Finally we 
select m\ as the index of the bin which contains the codeword u™(k) found and transmit it to 
nodes 2 and 3. 

C. Decoding at node 2 

Given x 2 and mi, we search in the bin B\ (mi) for an index k e [1 : 2 nRl ] such that: 

«.«?(*)) 6 7jJ st , lJa , e 3 > 0. (423) 

If there only one index that satisfies this we declare it as the index generated at node 1. If there 
several or none we choose a predefined one and declare an error. The chosen index is denoted 
as k( 2). 

D. Encoding at node 2 

Given x 2 and k( 2) we search for l e [1 : 2 nR 2 ] such that: 

(4,<(M2)),<(/,fc(2))) G 7^ 2t7lC72]e4 , e 4 > 0 . (424) 

If more than one index satisfies this condition, then we choose the one with the smallest index. 
Otherwise, if no such index exists, we choose an arbitrary one and declare an error. Finally we 
select m 2 as the index of the bin which contains the codeword u 2 (l, k( 2)) selected and transmit 
it to node 3. 

E. Decoding at node 3 

Given x%, v 2 and m 1 ,m 2 , the decoder search in the bins F>, (mi) and FFfm^) for a pair of 
indices (k, l) E [1 : 2 nRl ] x [1 : 2 nR 2 ] such that 

(j%,v2,vZ(k),v2(i,k)) e 7 e > 0 • ( 425 > 

If there only one pair of indices that satisfy this we declare it as the indices generated at node 1 
and 2. If there several or none we choose a predefined pair and declare an error. The chosen pair 
is denoted by (k(3),l(3)). Finally, the decoder declares (u^u^) = («"(fc(3)), u 2 (1(3), k(3))). 
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F. Error probability analysis 

Consider ( K,L ) the description indices generated at node 1 and 2, and (Mi,M 2 ) the corre¬ 
sponding bin indices. With K( 2) and (K (3), L(3)) we denote the indices recovered at nodes 2 
and 3. We want to prove that Pr { £ } < e' when n is sufficiently large, where 

£ = { (A7, X 2 ”, A s ”,V3*, 3)), 17 2 ”(L(3), ir(3))) t 7iW,vw l0a] «} ■ («6) 

We consider the following events of error: 

. A = |(Ar,A'j,A7,\/ 2 ”)^7;j lMaK](l }, £l > 0 . 

. £, = |(AT, Uf(k)) i T$ tUt]n V* 6 [1 : 2 "*]} , €2 > 0. 

• £3 — |i.V(, A' 2 . A'A. IA ,Uf(K)) £ 7(J lX2 _Y3i £ 2[/i]e 3 } > £ 3>0- 
. £4 = |3M 77. k 6 B,(A/,j, (AJ, [/,”(*)) 6 7^,^} , €3 > 0. 

. ft = {(A 2 ",r/ 1 "(A-(2)),C/ 2 “((,A-(2)))) i 7[J 2lJlt , 2|<4 We[l: 2"«>]} , e 4 > 0. 

. £b={(Ar,A?,AJ,^,C^(Jir),t^(A,A-(2))) ^7jJ lX2XaWlMl| ,}, f > 0. 

. £,= {3fc ,2 K, i=tL,k£ 0i(MO.i e B 2 (M 2 ), (aj, V 2 ", CTftfc), t/ 2 "(i, A))e7jJ sW3 j.}- 

Clearly £ C (Jj_, £,, In fact, it is easy to show that j(A'(3), A(3)) £ ! /\\ /.), K(2j A A'| c 
U i= 1 £i. From hypothesis, we obtain that lim, woc Pr {£ 1 } = 0. Choosing e± < u^j and e 2 < e c d 
we can use Lemma [5] and its Corollary (with the following equivalences: V = U \, A" = X\, 
(7 = 0 ) to obtain lim^noPr {£ 2 } = 0 if 

Ri > I(U 1 -,X 1 ) + S(e 1 ,e 2 ,€ C d,n) . (427) 

For the analysis of Pr {£ 3 } we can use Lemma[7j its Corollary and Lemma [ 8 ] defining Y = A 2 A 3 , 
X = X\ and U = U\ and using e 2 , e 3 and e c d sufficiently smalpjto obtain lirn^oo Pr {£<} = 0. 
For the analysis of Pr {£ 4 } we can write: 

Pr{£ 4 } = E [Pr{£ 4 |iC = k,Mi = m 4 }] 



f 



Pr < 

U {(*?,£/,”(&)) e TiWa} 

K=k 

Ml = 777-1 

> 


k^k 




K keBi(mi) 

j 



13 In the following, we will not indicate anymore the corresponding values of the constants e, the arguments of 6 and the 


equivalence between the involved random variables in order to use the lemmas from Appendix 


a 
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Using Lemma [ 6 ] (with the appropriate equivalences on the involved random variables) and the 
statistical properties of the codebooks, binning and encoding, we have, that for each k, mi. 


lim Pr 

n—>oo 


u 


x;,ui'(k) €T? XlUtla 


k^k,k£t3i(mi) 


K=k \ _ n 
Mi=mi f ~ u 


provided that 


-log E||5 1 (mi)|| < I(X 2 -,Ui) ^ S(e 1 ,e 3 ,e cd ,n) . 


n 


(429) 


(430) 


As E [||£>i(mi)||] = 2 n ^ 1 Rl ^> Vmi we have that lim, woo Pr {£ 4 } = 0 provided that 


Ri - Ri < I{X 2 - Ui) - 5 . (431) 

The analysis of Pr {£ 5 } follows the same lines of Pr {£_>}. The above analysis implies that 

Hm Pr { (a?, U?(K(2))) € = 1 . (432) 

Then, by Lemma [5] and its Corollary we have that lim^oo Pr {£ 5 } = 0 if : 

R 2 > I(X 2 -U 2 \Ui)+5 . (433) 

From Lemmas [7] and [ 8 j similarly as with Pr{£ 3 }, we have lim, woo Pr {£ 6 } = 0. Let us turn to 
analyze Pr {£ 7 }: 

Pr {£ 7 } = E [Pr (£ 7 !A' = k, L = l, Mi = mi , M 2 = m 2 }] 



r 



Pr < 

U { (aj, v?,u?(k), u;(i,k)) € i$, V2UlU2] ,} 

K=k,L=l 
Mi=mi ,M2=rri2 

> 


(k,l)^(k,l) 




fceZ3i(mi) 




„ ( k,i)eB 2 (m2 ) 

> 



< E [«! + a 2 + cr 3 ] (434) 


where we have 




|J { (a 3 “, Vf, U?(k), Ui(l, k)) 6 T? x ,v 2Vi v 2] <} 

K=k,L=l 

Mi=mi,M2=Tri2 

k^k 


k€Bi(mi) 


k (k,l)eB2(m 2 ) 

> 


(435) 
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\ 


a 2 = Pr < 

U 1 

'(xj, !£■,£/?(*), t/J(l>)) 

e T\X 3 V 2 U 1 U 2 ]e} 

K=k,L=l 
M\=mi ,M2=rri2 

> , (436) 


< ( k,l)eJ32(m2 ) 



j 

\ 


a 3 = Pr < 

U 1 


G ^[-X 3 F 2 C/it7 2 ]e} 

K=k,L=l 
Mi=mi ,M2=rri2 

> . (437) 


k^k,l^l 
k£t3i (mi) 

„ (fc,0sS 2 (m 2 ) 



> 



We can use the Lemma |6] to obtain: 


provided that 


— log E 
n 


— log E 
n 


— log E 
n 


Because on how the binning is performed, we have: 


E 


k : (k, l ) G B 2 (m 2 ), k G B^mi) 
E 


E 


which give us: 


(446), (447) can be combined with: 


R\ > R\ , 
R\ + R‘2 > R 2 , 


January 19, 2016 


103 = 0 

DO 

(438) 

< I(X 3 V 2 ] UiU 2 ) — 6, 

(439) 

< I(X 3 V 2 -, U 2 \Ui) — 5 , 

(440) 

< I(x :i v 2 - URJ 2 ) - 5 . 

(441) 

_ c^n^Ri—R\ — R 2 ) 

(442) 

_ 2 i n {R2—R2) 

(443) 

_ 2^-(^i+^ 2 — ^i — ^ 2 ) 

(444) 

u,U 2 )-5 , 

(445) 

U 2 \Ui) — 5 , 

(446) 

UiU 2 ) - 5 . 

(447) 

147]). Equations (|427 ), (431), 

(433), 


(448) 


(449) 
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which follow from the binning structure assumed in the generated codebooks. A Fourier-Motzkin 
elimination procedure can be done to eliminate R\ and R 2 obtaining the desired rate region 
(conditioning also the mutual information terms on Vi). ■ 

The following Corollary considers the case in which a genie gives node 2 the value of Mi. 
Indeed, this case will be important for our main result. 

Corollary 5: If a genie gives M x to node 2, the achievable region reduces to: 

R‘2 > I(X 2 - 1 U. 2 \X ? y x V 2 U x ) , (450) 

Ri + r 2 > iiXiX^u^xyy ). (451) 

The proof of this result is straightforward and thus it will not be presented. 

Appendix C 
Proof of Theorem Q] 

Let us describe the coding generation, encoding and decoding procedures. We will consider the 
following notation. With M^s.i we will denote the index corresponding to the true description 
U^ SI generated at node i at round l and destined to the group of nodes S e C (A4 ) with i S. 
With Mi_ts,i(j) where S e C (M), i ^ S, j e S wc denote the corresponding estimated index 
at node j. 


A. Codebook generation 


Consider the round l e [1 : K\. For simplicity let us consider the descriptions at node 1. We 
generate 2 ”^ 1-23 independent and identically distributed n-length codewords [/['; >2 3 ,z( m i->- 23 ,z, m w \,,,) 
according to: 




1 



n 

1->23,Z 


[Ui- 


-23,1 |VV[i j i ] ]e(l,23,Z) 



Tl 


[t/l-^23,1 IWj!,,]]£(!,23,Z) 



, e(l,23,/)> 0 (452) 


where mi^. 23,1 € [1 : 2 r ' 7?l ^ 2 3] and let mw-,, denote the indices of the common descriptions W-'J ^ 
generated in rounds t G [1 : Z — 1]. For example, mw (1 (] = {mi_>. 23 ,t, '^ 2 — 5 - i3,t,m 3 ^i 2it } l t= \. With 
w? x z j we denote the set of n-length common information codewords from previous rounds corre- 

tl(rP^ I \ 

sponding to the indices 7 n\\> [1 For each rn W - :i consider the set of 2 v 1 ^ 23 3 ^ 12 / codewords 

^ r r->23,z ( m i ->■ 23 ,z, ^ 3 ^- 12 ,z-i) m w [3 ;_!])• These n-length codewords are distributed independently 
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and uniformly over 2 nR( ^3 bins denoted by £>i_s.23,z (pi^-23,1, mw [3 with p 1^23,1 G [1 : 
2” fi '-'23]. Notice that this binning structure is exactly the same we used for the cooperative Berger- 
Tung problem in Appendix [Bj Node 1 distributes codewords f /['_^ 23 / (m i ^ 23 J- m 3 ^i 2 ,i-i, mw [3 (1] ) 
in a super-binning structure. This will allow node 2 to recover both, m 1 ^237 and m 3 ^ 12 , 1 - 1 , using 
the same procedure as in the Berger-Tung problem described above. Notice that a different super- 
binning structure is generated for every mw [3i _ ir This is without loss of generality, because at 

round l nodes 1, 2 and 3, will have a very good estimated of it (see below). 

6(0 6(0 

We also generate and 2 n ^i ->3 independent and identically distributed n -length code- 

words U^ 2l (mi^ 2 ,i,rn W[21v m V[12l l] ), and U^ 3l (mi^ 3 ,i,m W[2lv m V[13ll] ) according to: 




| U l->2.:,Z ^ ”0 


[Ul^2,l |W [2il] V [12>li !]]€(!, 2,0 l w|2,q> ^[12,1,1] 


/ I 77, / fl fl 

Vl^.llWp^V^^Ml^Z) l W [2,Z]’ V [12,Z,1] 


,(453) 


^S.Z^l-^S.Z. m w [2 , ;] , ™v [13 , u] ) 


1 ^ W 1-+3,Z G ^ 1 ^ 3 ,z|W [2>!] V [13 ,j,!]]€(!,3,Z) (^[2,Z] ’ ^[13,Z,1] 


/ r/ 


[^i- > 3,l|W[a,i]V [13 , I , 1] Ml ) 3 I Z) y^[2,Z] > ^[13,Z,1] 


,(454) 


where e(l, 2, Z) > 0, e(l, 3, Z) > 0, and m 1 _ >2 ,z G [1 : 2 nR i^] and ?ni_ >3ii G [1 : 2 n ^iis]. These 

codewords are distributed uniformly on 2 nR ^ n ~ bins denoted by 23]_>.2,z(pi->2,z, mw [2 n, m v [12; y) 

and indexed with pi^ 2 ,z G [1 : 2 n O-U] and on 2 n - R i^s bins denoted by (p^ 3 ,z, m W[2 l] , mv [13iiil] ) 

„(!) 

and indexed with 1 G [1 : 2" ' J, respectively. Notice that these codewords (which will be 
used to generated private descriptions to node 2 and 3) are not distributed in a super-binning 
structure. This is because there is not explicit cooperation between the nodes at this level. That 
is, node 2 is not compelled to recover the private description that node 1 generate for node 3, 
and for that reason the private description that node 2 generate for node 3 is not superimposed 
over the former. Notice that the binning structure used for the codewords to be utilized by node 
1 impose the following relationships: 


p(0 / £>(0 6(i-i) 

- fx l-s>23 ^ -‘H— >23 ' rl 3->12 t 

(455) 

d( 0 ^ p(0 

-Hu—>2 ^ -H-l—>2 , 

(456) 

£3 

CO 

A 

?a> 

I s 

00 

(457) 


The common and private codewords to be utilized in nodes 2 and 3, for every round, are generated 
by following a similar procedure and theirs corresponding rates have analogous relationships. 
After this is finished the generated codebooks are revealed to all the nodes in the network. 
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B. Encoding technique 


Consider node 1 at round l e [1 : K], Upon observing x r ( and given all of its encoding and 
decoding history up to round l, encoder 1 first looks for a codeword u'^ 23 z (mi_> 23j /, 'diw {1 ,, (1)) 
such that e c (l, 23, l) > 0, 



W [l,l] (1)), <^23,z(' m 1^23,Z, 


,](!))) e 7^ 1 _ >23ji X 1 W [1 ,q]e c (l !2 3,/) 


(458) 


Notice that some components in myy; [ 1 (] (l) are generated at node 1 and are perfectly known. 
If more than one codeword satisfies this condition, then we choose the one with the smallest 
index. Otherwise, if no such codeword exists, we choose an arbitrary index and declare an 
error. With the chosen index mi_> 23 ,i, and with rhy^ [3[1] ( 1), we determine the index pi_> 23i / of 
the bin 6 ^ 23,1 (pi^ 23 ,i,rh W[3l l] ) to which «?_> 2;M (mi-> 23 ,o™ 3 ^i 2 ,i-i(l),my^^l)) belongs. 
After this, Encoder 1 generates the private descriptions looking for codewords 
rhw [2A (1 ),m V[12M (l)),^ 3il (m^ 2)i , rh W[2 l] (1), m V[13il>1] (1)) such that 


(®i, ^[ 2 , 1 ] (mw [2A (l)),v][ 2M (m V[12M (1)), <_, 2i i(mi_, 2i i, m W[2l] (1), m V[12 , u] (1))) 

G 7 it/ 1 ^2.iWW [2ii] V [12ii , 1] ]6 c (l I 2,0; > ( 459 ) 

( X 1, W [2,l] (™W [2>I1 ( 1 ))ufl3,U](^V [13 , i , 1] (1)), <-, 3 ,i( m l^ 2 ,J> m W[Ji(] (1), m V[13iIil] (1))) 

G 7it7i^ 3i! X 1 W [2 , (] V [13 ,j ll] ]ec(l,3,0 > ( 46 °) 

respectively, where e c (l,2,Z) > 0 and e c (l,3,Z) > 0. Given (m W[2i] (1), m V[12 , M] (1), mv [13jM] (1)), 

the encoding procedure continues by determining the bin indices pi^. 2,1 and p\->:u to which the 
generated private descriptions belong to. Node 1 then transmits to node 2 and 3 the indices 
(pi^ 23 ,i,Pi^ 2 ,i,Pi^ 3 ,i)- The encoding in nodes 2 and 3 follows along the same lines and for that 
reason are not described. 


C. Decoding technique 

Consider round l e [1 : K + 1] and node 2. During round the present and previous round 
node 2 receives (pi_^ 23 ,i, j? 3 -»-i 2 ,i— 1 , Pi-»- 2 ,i, Pi-*- 3 ,i, 1 ,p 3 -»- 2 ,i— 1 )- However, only the indices 

(Pi_» 23 ,o P 3 —i>i 2 ,;— 1 , Pi—= > 2 ,uPz-> 2 , 1 - 1 ) are the ones relevant to him. Knowing this set of indices, node 
2 aims to recover the exact values of (mi_». 23) ;, m 3 _^i 2; j_i, m 3 _», 2j j_i). This is done through 
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successive decoding where first, the common information indices are recovered by looking 

for the unique tuple of codewords <^ 23 ,z( m i^ 23 ,o m 3 -> 12 ,i-i, (2)), u'l+nj-i(rn 3 ^ 12 , j_i, 

% [3l _i](2)) that satisfies: 

(2)), v [12,l,i]('" l V [12iiil] (2)), ^[23,/—1,3] ( 7 ^ Z V [23j j_ 1 ,3 ] (2)), 

<-+23,/( m i->23,0 m 3 -n2,t-i, (2)), «3_, 12)t _i(m3_,i 2)J _i, m W[3 ^ (2))) 

^ »23,i^3->12,!-l^'2W[3 i i_i]V[23 i (_i i 3]V[i2 i ( i l]]£dc(2,0 ’ ^rfc(2,/) > 0 (461) 

and also belong to the bins indicated by pi^. 23,1 and p 3 —i 2 .;-i • If there are more than one pair 
of codewords, or none that satisfies this, we choose a predefined one and declare an error. 
After this is done, node 2 can recover the private information indices by looking at codewords 

u i^ 2 ,i( m i^,i,m W[2 l] (2), m V[12A1] (2)) and « 3 _, 2) ,_ 1 (m 3 _, 2) i-i, rh W[l l] (2 ),m V[23 l _ h3] (2)) which sat¬ 
isfy 

{ X 2, (2)),^3,i-i,3](S 23 ,w,3](2))^ 

u i^2,i( m i^2,i, m W[2A (2), m V[l2ll] (2)), ( 2 ), ^v [23 ,*-i,3] ( 2 ))) 

G 7 [t/ 1 ^2, ! t/3^2,i-1^2>V [ 2,i]V [ 23, i _l,3]V [ i2,i,l]]e dp (2,0’ 6 d P {^J) > 0 (462) 

and are in the bins given by pi^ 2 ,z and p:\-r 2 i-\- If there are more than one pair of codewords, 
or none that satisfies this, we choose a predefined one and declare an error. The decoding in 
nodes 1 and 3 is exactly the same and for that reason are not described. 

D. Lossy reconstructions 

When the exchange of information is completed, each node needs to estimate the other nodes 
sources. For instance, node 1 reconstruct the source of node 2 by computing: 

^12 ,i = 9l2 {xii, v [12,A'+l,l]i, W[i,A'+l]i) , i = 1, 2,..., n, (463) 

and similarly, for the source of node 3: 

^13, i ~ 9l 3 ( x li, v [13,A'+l,l]i, W[i,A'+l]i) , % — 1, 2, . . . , 71. (464) 

Reconstruction at nodes 2 and 3 is done in a similar way using the adequate reconstruction 
functions. 
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E. Error and distortion analysis 


In order to maintain expressions simple, in the following when we denote a description without 
the corresponding index, i.e. U] i _^ sl or VV-'j n, we will assume that the corresponding index is the 
true one generated in the corresponding nodes through the detailed encoding procedure. Consider 
round l and the event T>i = QiD Ti, where for q > 0, 


Qi= {(A',”, XIXI WJj,, Vfi 2 . u] , V5 3A1| , Vfa#) 


eV 


[^'l^2^’3W[i ] i]V[ 1 2,i,l]V[i3 i | i i] V 123 ,l,2] ] e l 


,(465) 


Ti= 


| Mi^ s ,t(j) = s G C(M), i £ S, j e S, f e [ 1:1— l],with exception of 

-^3^-12,i-l ( 2 ), ^3—>2,Z— 1 ( 2 ) | • 


(466) 


The set Qi indicates that all the descriptions generated in the network, up to round / are jointly 
typical with the sources. The ocurrence of this depends mainly on the encoding procedure in 
the nodes. Set indicates, that up to round /, all nodes were able to recovers the true indices 

of the descriptions. This clearly implies that there were not errors at the decoding procedures 

in all the nodes in the network. The condition in Ti on M 3 _ 5 .i2 ) «-i(2), M 3 ^. 2 ,z-i( 2) is due to the 
fact, that the decoding of those descriptions in node 2 occurs during round l. The occurrence of 
T>i guarantees that at the beginning of round l: 

• Node 1 and 2 share a common path of descriptions W r \,, U V-" l21 , which are typical with 

. Node 1 and 3 share a common path of descriptions Wr",, U Vn 31 { which are typical with 

( V n V n 
1^1 5 ^-2 ’ ^-3 )• 

• Node 2 and 3 share a common path of descriptions VVSU V r ,‘ 2: >, 3 - which are typical 
with (X?,X2,X%). 

Let us also define the event Ef. 


Si ={there exists at least an error at the encoding or decoding in a node during round /} 

= 1J £enc(h 0 U Sdec(i, l ) (467) 

i&M 

where E enc (i, l ) contains the errors at the encoding in node i during round l and £d e c(i- 0 considers 
the event that at node i during round l there is a failure at recovering an index generated 
previously in other node. For example, at node 1 and during round l: 

E enc (i, l ) = Send 1, /, 23) U E enc { 1, /, 2) U E enc (l, l, 3) (468) 
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where 


Sendl, l, 23)=| ( X?, W^(M W[M (1 M W[l l] (1))) £ ^I^l^23,i^lW[ li j]]e c (l,Z,23) 


Vmi-^23,1 e [1 : 2 nR ^ 


(0 


(469) 


Ul. J, 2)= Xj\ Mf 2) q(M W[2>l] (l)), V^ 2 )M] (Mv [12>j>1] (1)), M W[2<1] (1), M V[12>1>1] (1)) 


s (i) 

^ ' 7 [C/i^2, i -YiVV [1 , !1 V [12 , iill ]e c (l,/,2) Vmi^2,z G [1 : 2 n !^ 2 ] 


(470) 


a. «, 3)= Xj\ Wf 2) q(M W[2>l) (l)), Vf 13)M] (Mv [13(J>1] (l)), M W[2<1) (1), M V[13>1>11 (1)) 


^ 7 [c/i^3, i XiW [1 , i] v [13 , i , 1] ]ec(i,i,3) Vrrzi^ 3iZ G [1 : 2 nR ^] 

Event Sdeci}, l) can be decomposed as: 


(471) 


£dec(j"i 0 u u 7 ^ . (472) 

seC(M),ies j-.jgs 

At the end of the information exchange phase we would expect the occurrence of V K+1 r\£ K+ i, 
where £k+i is the event of an error during round K 4-1. As during round K + 1 only node 2 
tries to recover the descriptions generated during round K in node 3, we have: 


£ l\ ■ 1 £dec (2, A + 1) — |m 3 ^i 2 i a'(2) 7 ^ M 3 _ 5 > i 2 ,a' or M 3 ^ 2 ,a'(2) ^ M 3 _ 5 > 2 ,a'| • (473) 

The occurrence of V K+ i D £k+\ guarantees that all the descriptions generated during the K 
rounds of information exchange in the network are jointly typical with the sources realizations 
and that those descriptions can be perfectly recovered in all the nodes. In this way, if we can 
guarantee that Pr \ V K+1 D £k+\ } -> 1, then with probability converging to one we obtain: 

• Node 1 and 2 share a common path of descriptions Wj K+l U Vj 2 K+l t which are typical 
with (X?,X2,X$). 

• Node 1 and 3 share a common path of descriptions Wj K+[ U Vj : > K+l , which are typical 
with (x?,X 2 ,xs). 

• Node 2 and 3 share a common path of descriptions Wj K+[ U VS 3 K+l 2 i which are typical 
with (x?,X 2 ,xs). 

Using standard analysis ideas, the average distortions (over the codebooks) at the reconstruc¬ 
tion stages in all the nodes satisfy the required fidelity constraints. From there is straight¬ 
forward to prove the existence of good codebooks for the network. In order to prove that 
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Vx{V K+ i fl £k+i } - > 1 let us write: 

L } n—>-oo 

Pr |^ir+i H ^x+i|=Pr {'Dr+i U £r+ 1} = Pr {£ > r+ 1} + Pr {^a'+i O £*r+i} 

<Pr [f) K+ i D V K } + Pr {'Dr} + Pr {V K+1 fl £r+\} 

<Pr {Vr} + Pr {V K+ 1 0 (V K fl £k) } + Pr {V K fl £k} + Pr {V K +i H £r+ 1 } 

K+l AT 

<Pr {V ,} + ^ Pr {D l nS l } + '52 Pr {V l+1 n (V t 0 £,) } . (474) 

i=\ 1=1 

Notice that 

Vi = { (AT, AT, X2) 6 7^*^} , ei > 0 • (475) 

From Lemma d| we see that for every e± > 0 Pr {V>i\ -> 0. Then, it is easy to see that 

Pr {77x4-1 fl £r+i } -> 1 will hold if the coding generation, the encoding and decoding 

procedures described above allow us to have the following: 

1) If Pr {Vi} -> 1 then Pr{£> m } -> 1 VI e [1 : K + 1], 

n —^00 n—> 00 

2) Pr {Vi n £ 1 } -> 0 VZ G [1 : K + 1], 

n—¥ 00 

In the following we will prove these facts. Observe that, at round Z the nodes act sequentially: 


Encoding at node 1 —>■ Decoding at node 2 —» • • • —» Encoding at node 3 —> Decoding at node 1. 


Then, using (467) we can write condition 2) as: 


Pr [V t n £,}=Pr (A n £ en c(l, Z)} + Pr {V t n £ dec (2, l ) 0 £ enc (l, Z)} 
+Pr [T>i n £ enc (2, l ) n £ enc (l, Z) n £ dec (2, l )} + 

• • • + Pr [v { n £ dec {i, i ) n £ enc (i, Z) n • • • n £ enc (3 , z)}. 


(476) 


Assume then that at the beginning of round Z we have Pr { Vi } - > 1. Let us analize the 

n—¥ 00 

encoding procedure at node 1. Let us consider Pr {T>i fl £ enc (l, 0}- We can write: 

Pr {Vi n £ enc (l, Z)}< Pr {£ enc (l, l , 23) n £>z} + Pr {4nc(l, Z, 2)nD,n £ en c(l, Z, 23)} 

+Pr {£ enc (l, Z, 3)nP,n 5 enc (l, Z, 23)} . (477) 

From Lemma [I] and the fact that lim„ ^ Pr {Qi} = 1 we have that Pr {Ai(l, 23)} = 1 

where: 

-4,(1,23) = {(AT, VVft,) 6 7i5 lVVll|]] „} , (478) 
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Then, we can use Lemma [5] to obtain: 

lim Pr {E enc (l, l, 23) D T>i} = 0 

n—>oo 

provided that 

R ( il 23 > I U^23,i\ W[i,q) + 4(1, l, 23) 
where 4(1, Z, 23) can be made arbitrarily small. On the other hand we can write: 

Pr { £ enc (1, Z, 2) n V l n £ enc (l, /, 23)} < Pr {S enc ( 1, Z, 2) n1, 2) n J)} + Pr {£,(1, 2)} (481) 
where 


( 479 ) 


(480) 


£z(4 2 ) - ( W, X 2 4 X 3 ", Vfi 2>i)1] , Vfc A l], Vfty-q) e 7 jJ lX 2 X 3 V V[ 2 iJ ]V [ i 2 ii| i ] V [ 1 3 ,!, 1 ] V [ 23 ,!, 2 ] ]ei(l, 2 ) 

(482) 


where ej(l,2) > 0. As explained before, Pr {£?;}-» 1. Then, from condition (480) we have 

n—>-oo 

Pr {£„*(!, (, 23) n 7i} = Pr {(X,", Wf y] ) e V.Ww.MU*,) 


-> 1 . 


(483) 


Moreover, from the coding generation and the encoding procedure proposed is immediate to use 
Lemma [8] to show that: 


Pr (( 7 ( 4 . 23 ,/ ~ U l^- 23 ,l\ X l , W [l:l]i ^-enc(l, Z, 23) fl 7|) — 

1 { M 1-a 23,Z G ^n.23,!|A-iW[i il] ]e c (l,23,0 ( X l’ W fl,Z] / 


'T'n [ , r n „.,n 

' lUi^23,l\XiW lltl] ]e4l,23,l) W [l,l] 


(484) 


Then, from Markov chain 


(485) 


^l-S>23,Z ~ e ~ (3fi, Wfi^]) -e- (X2,X3,V[12,Z,1],V[13,Z,1],V[23,Z,2]) 

and the Markov Lemma [7j for sufficiently small (e c (l, Z, 23), ei, ez(l, 2)) and after some minor 
manipulations, we can obtain: 

Pr {£)(!, 2)}-> 1 . (486) 


From equation (481) it is clear that we need to analyze term Pr (£ enc (l, Z, 2) DQid, 2) D J 7 /}. 
Similarly as before lim n _ > . 0O Pr{^l.z(l, 2)} where: 


•4(1,2) = {(X t »,Hf i( ,VJ, i0] ) € , 


(487) 


January 19, 2016 


DRAFT 













85 


which allow us to write: 


Pr {£ enc ( 1, /, 2) n 0,(1, 2) n 7i} < Pr (£ enc (l, Z, 2) n Ml, 2) n x,} . (488) 

Using again Lemma [5l we obtain that Pr (£ enc (l, l, 2) fl ^(1, 2) D X{\ -> 0 provided that 

| f 71—KX) 

R\% 2 > I (x i; 17^2,t W[ 2 ,i]V[i2,Z,l]) +S C {1,1,2) 


(489) 

where <5 C (1, /, 2) can be made arbitrarly small. For the analysis of Pr {£ enc (l, l, 3) fl U/ fl £ enc (l, l, 23)} 
we follow the same procedure. We can write 

Pr {£ enc (l, l, 3)nv l n £ enc ( 1, l, 23)} < Pr {£ enc (l, l, 3) n Gi( 1, 3) n J)} + Pr {£(1, 3)} (490) 
with 


> W [2,i]^[12,1,2]) ^[13,/,1]) ^[23, 1 , 2 ]) £ A' 2 X 3 VV[2,i]V [12 , !j2] V [13>!jl] V [23 , !i2] ]ej(1,3) 

(491) 

Using the Markov chain 


Ul^.2,1 -e_ {Xi, W[2,Z], V[12,Z,1]) ~ e_ (-^2j -^"37 '^[13,i,l]'^ 7 [23,i,2]) j 


(492) 


the fact that Pr{(^(l,2)} -» 1 and the Markov Lemma 7 and Lemma [8] for appropriately 

71—>00 

chosen values of (e c (l, l, 2), e/(l, 2), e/(l, 3)) we have: 


Pr{ft(l,3)}->1 . 

71—>-00 

Following exactly the same reasoning as above, we have that in order to have 

Pr{fenc(l,i,3)nX> i n5 cnc (l,Z,23)} -->0 , 


besides conditions (480) and (489) we need: 


R\ l l :i > I [X.-JJ, 


; l->3 ,1 


V[i 3j i,i] ] + <5c(l, l, 3) 


(493) 


(494) 


(495) 


for sufficiently small 5 C (1, l, 3). With these conditions we have proved that the encoding procedure 
in node 1 during round l permit us to have: 


Pr {Ri D £ enc (l, Z)}-» 0 . 

71—^OO 

Another instance of the Markov lemma, jointly with Markov chain 

(4—>3,Z (^1, W[2,i], V[i3 ; i ? i]) -e- (X 2 , X 3 , V[12,Z,2]V[23,Z,2]) 


(496) 


(497) 
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and Lemma [8] allow us to have: 


Pr {0,(2,13)}-> 1 

n—>-oo 

where 


( 498 ) 


fit 2.13) = € 7g w ^^v^w.W«(M»} • 

(499) 

At this point we have to analyze the decoding in node 2. If that decoding if successful, with 
(498), the analysis of the encoding at node 2 follows the same lines as abovep) The same can 
be said of the encoding at node 3 (after successful decoding). In this way, we terminate round 
l with 


Pr{D m } = Pr{Gi +1 fl J)+i}- y 1 


(500) 


which is one the results we wanted. Clearly, analyzing now the decoding at node 2 (from which 
we can easily extrapolate the analysis to the decoding at node 1 and 3) we will be able to obtain 
Pr {V>i n E \} - y 0 which is the other required result. 

n—>■ oo 

The decoding in each of nodes follows the approach of successive decoding. Decoder 2 will try 
to find first the common descriptions Mi_> 2 3 ,z and M 3 _> 12 , 1 - 1 - Then, it will try to find the private 
descriptions AT] _> 2 v and M 3 ^. 2 ,z-i (using of course the previously obtained common descriptions 
as side information). Clearly, the use of joint-decoding could improve the rate region. However, 
the analysis of this strategy, besides of being more difficult to analyze, it will give rise to more 
complex rate region. It can be easily seen, that the joint-decoding region will contain several 
sum-rate equations that will contains common and private rates. Successive decoding allows for 
a rate region where the sum-rate equations contains solely common rates or private rates, being 
more easy to analyze and understand. 

In order to analyze the decoding, we can write: 


Pr {Vi n £ dec {2 , l ) n £ enc (l, 0}<Pr {£ dec (2, Ijn^D 0,(2,13)} + Pr {0,(2,13)} . (501) 


As Pr{0,(2,13)} - y 1 we can concentrate our effort on the first term. Event £ dec (2,l) can 

71—KX) 

be written as: 


£ded 2, i) = n common (2, l) U H p rivate{2 , l) 


(502) 


4 See that Qi{2, 13) has, for the encoding at node 2 the same role that Qi has for the enconding at node 1 during round l. 
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where 

’Hcommon(2,/) = | ^M3^12,i-l(2), Ml_s>23,z(2)^ 7 ^ (-^3^-12,Z-l, ^1^-23,/) j , (503) 

"Hpriuote(2, i) —| ^-^3-»-2 J Z-l(2), Mi^.2,z(2)^ 7 ^ {M^-^2,1-1 , -Wl-»-2,z) j • (504) 

From these definitions, we can easily deduce that: 

Pr {£dec(2,0 n Ti fi Qi( 2,13)}=Pr {'H common (2, Z) n J-) Cl Qi( 2,13)} 

+Pr { 'Hprivate (2, /) fi J-) fi (7/(2,13) fl H common ( 2 , 0 } 
<Pr{ZC„ mmm (2, ()} + Pv{IC rrM J2,l)} (505) 


where 


A3 C0mm0 n(2, /) — |3(mi_>.23,Z, ^3->12,Z-l) 7 ^ (-^L-l-237, -^3-712,Z-l), (wtl->23,Z, ^3->12,Z-l) 
G £> 1 —>23,Z (-Fl->23,z) x ^3->12,Z-l(-f3->12,Z-l) : (^2 ) ^[3,Z-l]i ^[23,1-1,3]) ^[12,Z,1]) 


^lU23,z( m 1^23,Z, ™3^12,Z-l), ^3->12,Z-l( m 3^12,Z-l) ^ 7[XiW [2! ; ] V [12 , ; , 1] V [23 ,i_ l! 3 ] ]e dc (2,Z) J } >(506) 

ZC private (2, l ) |3(mi^2,i, m 3-,2,;-l) 7^ ((V7l_,2,it ^3->2,i-l), (wi-,2,;, ^3-,23-l) 

6 IWA-w) X 1 W->(*Wi) : (A'J.Wgj!, Vg 3|( _ li 3], V^u,, 

f/ r-42,z( m 1^2,z),^-42,z-l( m 3^2,z-l) £ ^\XiW [2<l] V [ 12 , i , 2 ] V [ 23 ji , 2 ] ]e dp (2,Z)) } ,(507) 


where e<fc(2, /), e: r / p (2,1 ) are carefully choseij^J and for a saving of notation we considered only the 
indices to be recovered, i.e., ^ 3 -^ 12 ,i-i) = ^ 237 (^ 1 ^ 237 , ^ 3 ^ 127 - 1 , ) 

Consider first the recovering of the common information. Node 2 has to recover two indices from 
a binning structure as the one in the cooperative Berger-Tung problem described in Appendix [B] 


In Fig. [TOj we have a representation of the problem seen at decoder 2. Node 3 generate 
a common description at rate using W7, as side information. Similarly node 1, 

15 Using Lemma [l] to have: 

<?i(2,13)C^Y 2 , W[ 2 ,I], V[2 3 ,i_1,3], V[ 1 2 , 1 , 1 ]) G 7[J 1 vv [2ii] v [12i! , 1] v [2 3, i _ li3] ]e < j 0 (2,o) , 

a (2, 13 )c(^2 , W[2,i], V[2 3 ,I,2], V[12,!,2]) G lix 1 W [2i , ] V [12] i i2] V [23i , )2] ]e # (2,!)) ■ 
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Figure 10: Cooperative Berger-Tung decoding problem for node 2. 


after decoding the common description from node 3, generate its own description using the 

recovered one and also as side information. All these operations all done using the 

super-binning structure as in the cooperative Berger-Tung problem in Appendix [B] Then, node 
2 , using (X 3 , Wpj.j], V[" 2n] , V[ 2 3Z _ 13] ) as side information tries to recover the descriptions 
generated at node 3 and 1. Remember the fact that the encoding procedure at nodes 1 and 3 
requires: 

RiZu > I{X 3 ] u 3 ^ i2.i-ilWt3.i_i]) + 4(1 ,1 - 1,12) , (508) 

>23 > I{X\ ; Ui^.23,1 |W m ) + «1,(,23) , (509) 

M'X < A® 23 + (510) 


and that the following Markov chains: 

Un—*12,1—1 (X 3 , W[3,J_1]) -©- (Xi, X 2 , V[i2,i,l], V[23,Z—1,3]) , (511) 

Ui^.23,1 (Xi, VVf!,/]) -©- (x 2 , X 3 , V[i 2 ,i,l], V[23,Z,2])/, (512) 

are implied by the Markov chains in the conditions of Theorem |T] In this way, we can use the 
results in Appendix B to show that the following rates imply Pr {JC comrnon (2, /)} - 


A 0: 


Rfl 23 > £4-^23,/|^ 2 >V [ i i qV [ 2 3 ,i-l |3] V [1 2 ,i ll] ) + 4c( 2 , l) , 


(513) 


4°>23 + 44i2 > /(X 1 X 3 ;E/ 1 _ 23il C/ 3 -. 12l i- 1 |X 2 >V [3 , i _i ] V [23 , l _ 1)3] V [1 2 )l) i ] ) + ^(2,Z) ,(514) 
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Figure 11: Berger-Tung decoding problem for node 2 when it tries to recover the private 
descriptions generated in nodes 1 and 3. 


where Sd c (2,l) 1 S' dc (2 1 l) can be made arbitrarily smalj^J 

The decoding of the private descriptions can be seen as a standard Berger-Tung decoding 
problem (see Fig. |TT]) where the binning used to transmit the descriptions generated in node 3 
and 1 is not cooperative (in the sense of Theorem [9]) as in the case of the common descriptions. 
Lemma [6] can bee easily used to analyze Pr {JC private (2. /)}. The following conditions guarantee 
that Fr{IC private (2 ,l)} -> 0: 

n—>■ oo 

l6 Here we considered the Corollary to Theorem]^ That is we assumed, that node 1 knows perfectly the value of 
This follows from the assumed fact, that at the beginning of round l, the probability of decoding errors at previous rounds in 
all nodes is goes to zero when n —> oo. In this way, the constraint on rate f?3_>i2,i-i that should be considered, according 
to Theorem [5] is not needed. In fact, constraints on rate -R 3 -u 2 ,(-i will arise when at node 1 we consider the recovering of 
A/3_>i2,! —1 and Af2->i3,t-i- For that reason, the analysis carried on is valid. Through this analysis we avoid carrying a lengthy 
and difficult Fourier-Motzkin procedure to eliminate ^1^231^2^131^3^12 f° r I = [1 : K], 
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I (^i-, 2 I i;^ 2 V [23) i ) 2 ] |w [2) qV [12i , il] ) - 5 dp (2,l) (515) 

R { 3^S<R { sZS + I (u 3 ^ . 2 , 1 - 1 ; -Xr22/i^23,iV [ i2, i , 2] |>V [1 ,i ] V [2 3,,_i,3]) - 4(2, /) (516) 

44 + 44 2 <44 + 44 + / (Ul^2,h X 2 V[23,l,2] | W[ 2 ,i] V [12ji>1] ) 

+-f (Uz-+2, i—1; ^2^1-5.23,iV[i2,/,2] W[1,/]V[23,Z-1,3]) 

— I (u 3 ^ 2 ,l-i; Ui^ 2 ,l W[2,/]V[23,Z-1,3]V[12,/,1]-Y2^ — 8 dp (2,l) (517) 

where M2, Z), ^(2,0 ,S d (2,1) can be made arbitrarily small. Then, combining all the obtained 
results, we have that: 

Pr {Ri n S dec (2, l) n S enc (l, ()} -> 0 . (518) 

n —>-oo 

At this point, the story is as it was at the encoding stage in node 1 and all the steps can be 
repeated with minor modifications, proving the desired results at the end of round l: 

Pr {Ri+i} -> 1, Pr {Di n £{\ ->0. (519) 

n—» oo n—too 

The other rates equations are as follows: 

. Encoding at node 2: 

44 13 > I (x 2 ] U 2 ^i3,i\ W[ 2 ,q) + 8c(2, l, 13) (520) 

44i > I (x 2 - |>V [3>i] V [12>i>2 ]) + S c (2, l, 1) (521) 

44 3 > I (*2] U 2 ^3, l \W[3,l]V[23,l,2}) + S c (2, l, 3) (522) 

• Decoding at node 3: 

44ia > I (x 2 - U 2 ^ 3 ,i\x 3 yV {2A V m , i]V[23,i > 2]) + M3, l ) (523) 

44 13 + 44 23 > I {x^- f/ 1 ^23,Z^13,z|X 3 W [1 ,z ] V [1 3,Z,l]V [ 23,Z,2]) + C(3, l ) (524) 
44 3 < 44s + 1 {y 2 ^ 3,1 ; -^3 V[i3 ; f,3] | W[3,Z] V[23,Z,2] j — 5<Zp(3, /) (525) 

44s < 44s + 7 (^3 ) i;^3^2-,13 ) tV [ 23 I t ) 3]|w [ 2 i qV [13) i,l]) - 4(3, 0 (526) 

44 3 + 44s < 44s + 44s + / (tw | 

+ -f ^l-^3,z; -^3^2-»-13,iV[23,J,3] 

— I (Ui^. 3 /i U 2 ^3,l |w [ 3,z]V [2 3 i z > 2]V [1 3 > z,i]X 3 )-4(3,0 (527) 


January 19. 2016 


DRAFT 



91 


Encoding at node 3: 


rV\ 12 > I[X 3 -,U^ 12 ,1 


A?ii > I (^ 3 ; U, 


3, <^3^-1 ,l 


W[3,z]J+<5 c (3,/,12) 

W[i ) i + i]V[i3 i / ! 3] N ) + S c ( 3, l, 1) 


Rll 2 >i(x 3 -u 3 ^ l V[23,Z,3] J +^c(3,/,2) 

Decoding at node 1: 

Ri\ 12 > I (Xz-U^ m AiW[3 i z]V[i2,Z,2]V[i3 i i ) 3] j + 5dc{ 1,0 


(528) 

(529) 

(530) 

(531) 


D 1 

it 


(0 

3—^12 


+ R 


(0 

2—1!3 


> / ^A" 2 A" 3 ; (/2-s>13,1^3^12,/ |^ 1 W [ 2 l/] V [1 2,, I 2]V [1 3 I Z,3]) + 4(M) (532) 


< Rl 4i + / ( £/3->i,z; V [12> I+1 > 1 ] W [li , +1] V [13) z I 3] ) - 5dp(l, 0 (533) 


?(0 


R 


(i) 

2-s-l 


< f?.^, + I ( U 2 ->i,i] A 1 Z/ 3 - + i2 1 zV [ i3 I i + i,i] Wp.qVfia.z,^ ) - 4(1, 0 (534) 


i? 


(0 

2—rl 


+ «™i<fi2ii + flail + 0^1,1 ; AiV[i2,z+i,i] I VV[i^_|_i] V[i3 5 z ; 3] 


>(0 


,(0 


+/ ^f/2-»l,d Aif/3^i2 ) iV[i3 i z + i i i] W[ 3j qV[i2,Z,2] 

—I (u 2 ^i,f, U 3 ^,i |w [1 , l+1] V [1 2 ) Z,2]V [1 3,J,3]^ 1 ) “4(1,0 • (535) 
The final private rate equations in Theorem |Tj follows from a rather simple Fourier-Motzkin 


elimination procedure [22], 
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